Fluent Bit Provisioning

Laralord provisions Fluent Bit, a lightweight and high-performance log processor and forwarder, to collect, filter, and ship logs from Kubernetes clusters to OpenSearch. Integrated with Laravel applications for centralized logging, Fluent Bit enables efficient log management without a dedicated UI, relying on OpenSearch Dashboards for visualization. This page details the Fluent Bit deployment, its configurations, log access via OpenSearch Dashboards, and integration with Laralord’s multi-tenant SaaS platform. For more information, refer to the official Fluent Bit documentation.

Fluent Bit Overview

Fluent Bit is an open-source log processor and forwarder designed for high performance and low resource consumption. It collects logs from Kubernetes containers and system services, filters and enriches them, and forwards them to OpenSearch for storage and analysis. Laralord deploys Fluent Bit to enable centralized logging for multi-tenant SaaS applications, with logs visualized in OpenSearch Dashboards, secured via SSO and mTLS.

Fluent Bit Deployment Details

Laralord deploys Fluent Bit in a user-defined namespace using a Helm chart from the Fluent project. The deployment is configured to collect, filter, and forward logs to OpenSearch, with RBAC settings for Kubernetes access. Key aspects include:

• Namespace: Deployed in a user-defined namespace, isolating Fluent Bit from other services for security and organization.
• Helm Configuration: Uses a specific Helm chart version with custom input, filter, and output configurations tailored for Kubernetes logging.
• Inputs: Collects logs from container files (/var/log/containers/*.log) with a tail input and from kubelet systemd logs, using docker parsers and multiline support.
• Filters: Enriches logs with Kubernetes metadata, excludes OpenSearch and Fluent Bit pod logs, and enables log parsing and exclusion for efficiency.
• Outputs: Forwards logs to OpenSearch via HTTPS on port 9200, with Logstash formatting, TLS enabled (verification disabled), and retry limits for reliability.
• RBAC Settings: Enables RBAC with node and event access, allowing Fluent Bit to collect cluster-wide logs and metadata.
• Resource Efficiency: Configures a 50MB memory buffer limit and a 10-second refresh interval to optimize performance.
• Log Access: No dedicated UI; logs are visualized in OpenSearch Dashboards, accessible via a secure APISIX Gateway.

Key Features

The Fluent Bit deployment by Laralord provides the following features:

• Log Collection: Gathers logs from Kubernetes containers and systemd services (e.g., kubelet) for comprehensive monitoring.
• Log Filtering: Enriches logs with Kubernetes metadata and excludes irrelevant logs (e.g., OpenSearch, Fluent Bit pods).
• OpenSearch Integration: Forwards logs to OpenSearch with Logstash formatting for efficient storage and querying.
• Resource Efficiency: Lightweight design with low CPU and memory usage, suitable for large-scale clusters.
• Secure Forwarding: Uses TLS for log transmission to OpenSearch, with configurable retry limits for reliability.
• Laravel Compatibility: Supports centralized logging for Laravel applications, visualized in OpenSearch Dashboards.

Why Fluent Bit?

Fluent Bit is chosen by Laralord for its performance, simplicity, and compatibility with Kubernetes and OpenSearch ecosystems. Key advantages include:

• High Performance: Lightweight and efficient, minimizing resource usage while handling high log volumes.
• Kubernetes-Native: Optimized for Kubernetes, with built-in support for container and systemd logs, enriched with metadata.
• OpenSearch Synergy: Seamlessly integrates with OpenSearch for log storage and visualization in OpenSearch Dashboards.
• Flexibility: Supports customizable inputs, filters, and outputs, tailored for multi-tenant logging needs.
• Open-Source: Backed by a strong community, ensuring reliability and ongoing development.

Integration with Laralord

Fluent Bit is a critical component of Laralord’s multi-tenant SaaS platform, providing centralized logging for tenant applications and infrastructure. Key integrations include:

• Tenant Logging: Collects tenant-specific logs, forwarded to OpenSearch for isolated storage and analysis.
• Laravel Applications: Captures application logs from Laravel containers, enabling debugging and monitoring via OpenSearch Dashboards.
• OpenSearch Integration: Forwards logs to OpenSearch, with visualization and querying in OpenSearch Dashboards, secured via SSO and mTLS.
• APISIX Gateway: Leverages APISIX for secure access to OpenSearch Dashboards, where Fluent Bit logs are visualized.
• Argo CD Synergy: Monitors Argo CD-managed applications by collecting deployment-related logs for analysis.
• Prometheus + Grafana Complement: Enhances observability by providing searchable logs alongside Prometheus metrics and Grafana dashboards.
• Database Logging: Captures logs from PostgreSQL, MySQL, and Valkey, indexed in OpenSearch for unified analysis.

Log Access via OpenSearch Dashboards

Fluent Bit has no dedicated UI, with logs accessed through OpenSearch Dashboards, secured via Laralord’s Single Sign-On (SSO) proxy and APISIX Gateway. Key features of the log access mechanism include:

• OpenSearch Forwarding: Logs are sent to OpenSearch with Logstash formatting, stored under the fluent-bit prefix for easy querying.
• SSO Authentication: Users access OpenSearch Dashboards via Laralord’s frontend, leveraging SSO for authenticated log visualization.
• mTLS Encryption: Dashboard traffic is encrypted using mutual TLS (mTLS), requiring a client certificate managed by Laralord.
• APISIX Gateway Proxy: Routes OpenSearch Dashboards requests to a custom domain, enforcing SSO, mTLS, and HTTP-to-HTTPS redirection.
• Log Visualization: OpenSearch Dashboards enables users to search, filter, and visualize Fluent Bit logs, including Kubernetes metadata and application logs.