Configuring and installing the plugin

Rakuten recommends installing td-agent version 4. However, you can also install the community Fluentd agent. For more information about installing Fluentd, see Overview.

Downloading and installing the SixthSense Fluentd plugin

  1. Download the latest SixthSense plugin (.gem file) from gitlab artifactory using the https://artifacts-observability.sixthsense.rakuten.com/sixthsense/sixthsense-log-monitoring/-/packages URL. Credentials will be provided by the onboarding team.

    The gem file package is available at https://artifacts-observability.sixthsense.rakuten.com/sixthsense/sixthsense-log-monitoring/-/packages/9629.

  1. Download the fluent-plugin-sixthsense-1.2.3.gem file and configure the agent.
curl --header "PRIVATE-TOKEN:{TOEKN}" --output fluent-plugin-sixthsense-1.2.3.gem "https://artifacts-observability.sixthsense.rakuten.com/api/v4/projects/33/packages/generic/sixthsense-fluentD-agent/1.2.3/fluent-plugin-sixthsense-1.2.3.gem"

Installing SixthSense agent

If you installed fluentd from source or from gem, then plugins can be installed using the following gem command in the folder where the .tar or .zip files are available.

gem install fluent-plugin-sixthsense-1.2.3.gem

Windows

If you used Windows while installing fluentd, use the following command.

winget install td-agent

Adding SixthSense Agent plugin to td-agent

If you used the td-agent packaging while installing fluentd from DEB, RPM, .msi and so on use the following command in the folder where the .tar or .zip files are available.

/opt/td-agent/bin/fluent-gem install fluent-plugin-sixthsense-1.2.3.gem

Fluentd configuration file

Example:

<source>
@type tail
read_from_head true
path /var/log/containers/*sixthsense*.log
pos_file /var/log/fluentd-containers1.log.pos
exclude_path ["/var/log/containers/fluent*"]
path_key filename
<parse>
@type multiline
format_firstline /\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s-/
format1 /(?<message>.*)/
</parse>
tag kubernetes.*
</source>
<filter kubernetes.**>
@type record_transformer
<record>
log_name ${tag}
hostname "#{Socket.gethostname}"
</record>
</filter>
<match **>
@type sixthsense
api_key (SixthSense_Token must be fetched from SixthSense dashboard)
base_uri https://http-collector-observability.sixthsense.rakuten.com/log/v1/log
# For connectivity to backend using onecloud private network: https://sixthsense-backend.jpe2-caas1-prod1.caas.jpe2b.r-local.net/log/v1/log
# For connectivity to backend using intra public network: https://observability-backend-core.sixthsense.rakuten.com/log/v1/log
service_name (Application name)
</match>

In the above, example, there are three sections as follows:

Source

In the example we are using tail plugin. The tail plugin is configured to read all the logs in the /var/log/containers/ directory. The position of the log file till which parsing is completed, is stored in the pos_file.

In the mandatory parse section, the multiline plugin is used for parsing stack traces which can span multiple lines. If this is not required, you can disable parsing using the none plugin

<parse>
@type none
</parse>

Filter

In the filter section, all logs tagged with kubernetes are processed. The record_transformer plugin is used for enriching the data with log_name, and hostname.

<filter kubernetes.**>
@type record_transformer
<record>
log_name ${tag}
hostname "#{Socket.gethostname}"
</record>
</filter>

Match

In the match section, SixthSense plugin is used. Here there are 3 fields:

api_key - The api key needs to be copied from the settings page in the SixthSense dashboard. base_uri - The SixthSense endpoint url. (currently pointing to satellite server directly) service_name - The service_name is the name given for the agent which will be displayed in the SixthSense dashboard with this name.

<match **>
@type sixthsense
api_key (SixthSense_Token must be fetched from SixthSense dashboard)
base_uri https://http-collector-observability.sixthsense.rakuten.com/log/v1/log (for OneCloud Plantform, use base_uri https://sixthsense-backend.jpe2-caas1-prod1.caas.jpe2b.r-local.net/log/v1/log)
service_name (Application name)
</match>

The services are now listed and displayed in the Log Monitoring dashboard.