Install by RPM Package (Red Hat Linux)

Installing td-agent

  1. Install from rpm Repository

    Note: If your OS is not supported, consider gem installation instead.

Red Hat / CentOS

Download and execute the the following install script with curl:

# td-agent 4
$ curl -L https://toolbelt.treasuredata.com/sh/install-redhat-td-agent4.sh | sh

Executing this script will automatically install td-agent on your machine. This shell script registers a new rpm repository at /etc/yum.repos.d/td.repo and installs td-agent. We use $releasever for repository path in the script and $releasever should be the major version only like "7". If your environment uses some other format like "7.2", change it to the major version only or set up TD repository manually.

Amazon Linux

For Amazon Linux 2:

# td-agent 4
$ curl -L https://toolbelt.treasuredata.com/sh/install-amazon2-td-agent4.sh | sh
  1. Launch Daemon.

td-agent provides two scripts:

systemd

Use /usr/lib/systemd/system/td-agent script to start, stop, or restart the agent:

$ sudo systemctl start td-agent.service
$ sudo systemctl status td-agent.service
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
Loaded: loaded (/usr/lib/systemd/system/td-agent.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2022-09-27 06:14:36 UTC; 1s ago
Docs: https://docs.treasuredata.com/display/public/PD/About+Treasure+Data%27s+Server-Side+Agent
Process: 33953 ExecStart=/opt/td-agent/bin/fluentd --log $TD_AGENT_LOG_FILE --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=0>
Main PID: 33959 (fluentd)
Tasks: 10 (limit: 4958)
Memory: 78.1M
CGroup: /system.slice/td-agent.service
├─33959 /opt/td-agent/bin/ruby /opt/td-agent/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid
└─33962 /opt/td-agent/bin/ruby -Eascii-8bit:ascii-8bit /opt/td-agent/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td>

To customize systemd behavior, put your td-agent.service in /etc/systemd/system.

Note: In td-agent 4, the path is different i.e. /opt/td-agent/bin instead of /opt/td-agent/embedded/bin.

init.d This is for CentOS 6, non-systemd based system. Use /etc/init.d/td-agent script to start, stop, or restart the agent:

$ sudo /etc/init.d/td-agent start
Starting td-agent: [ OK ]
$ sudo /etc/init.d/td-agent status
td-agent (pid 21678) is running...

The following commands are supported:

$ sudo /etc/init.d/td-agent start
$ sudo /etc/init.d/td-agent stop
$ sudo /etc/init.d/td-agent restart
$ sudo /etc/init.d/td-agent status

Please make sure your configuration file path is:

/etc/td-agent/td-agent.conf
  1. Post Sample Logs via HTTP.

The default configuration (/etc/td-agent/td-agent.conf) is to receive logs at an HTTP endpoint and route them to stdout. For td-agent logs, see /var/log/td-agent/td-agent.log.

You can post sample log records with curl command:

$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test
$ tail -n 1 /var/log/td-agent/td-agent.log
2018-01-01 17:51:47 -0700 debug.test: {"json":"message"}

Installing calyptia-fluentd

  1. Install from rpm Repository It is highly recommended to set up ntpd on the node to prevent invalid timestamps in the logs. Note: If your OS is not supported, consider gem installation instead.

Red Hat / CentOS

Download and execute the install script with curl:

# calyptia-fluentd 1
$ curl -L https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-redhat.sh | sh

Executing this script will automatically install calyptia-fluentd on your machine. This shell script registers a new rpm repository at /etc/yum.repos.d/Calyptia-Fluentd.repo and installs calyptia-fluentd.

We use $releasever for repository path in the script and $releasever should be the major version only like "7". If your environment uses some other format like "7.2", change it to the major version only or set up TD repository manually.

CentOS Stream

For CentOS Stream 8:

# calyptia-fluentd 1
$ curl -L https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-centos-stream.sh

Amazon Linux

For Amazon Linux 2:

# calyptia-fluentd 1
$ curl -L https://calyptia-fluentd.s3.us-east-2.amazonaws.com/calyptia-fluentd-1-amazon-2.sh | sh
  1. Launch Daemon.

calyptia-fluentd only provides systemd's unit file:

systemd

Use /usr/lib/systemd/system/calyptia-fluentd script to start, stop, or restart the agent:

$ sudo systemctl start calyptia-fluentd.service
$ sudo systemctl status calyptia-fluentd.service
● calyptia-fluentd.service - calyptia-fluentd: Fluentd based data collector for Calyptia Services
Loaded: loaded (/usr/lib/systemd/system/calyptia-fluentd.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2021-05-31 01:37:47 UTC; 4h 38min ago
Docs: https://docs.fluentd.org/
Process: 694 ExecStart=/opt/calyptia-fluentd/bin/fluentd --log $CALYPTIA_FLUENTD_LOG_FILE --daemon /var/run/calyptia-fluentd/calyptia-fluentd.pid $CALYPTIA_FLUENTD_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 1365 (fluentd)
CGroup: /system.slice/calyptia-fluentd.service
├─1365 /opt/calyptia-fluentd/bin/ruby /opt/calyptia-fluentd/bin/fluentd --log /var/log/calyptia-fluentd/calyptia-fluentd.log --daem...
└─1368 /opt/calyptia-fluentd/bin/ruby -Eascii-8bit:ascii-8bit /opt/calyptia-fluentd/bin/fluentd --log /var/log/calyptia-fluentd/cal...

To customize systemd behavior, put your calyptia-fluentd.service in /etc/systemd/system.

Please make sure your configuration file path is:

/etc/calyptia-fluentd/calyptia-fluentd.conf

  1. Post Sample Logs via HTTP.

The default configuration (/etc/calyptia-fluentd/calyptia-faluentd.conf) is to receive logs at an HTTP endpoint and route them to stdout. For calyptia-fluentd logs, see /var/log/calyptia-fluentd/calyptia-fluentd.log.

You can post sample log records with curl command:

$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test
$ sudo tail -n 1 /var/log/calyptia-fluentd/calyptia-fluentd.log
2021-05-31 06:19:04.415878392 +0000 debug.test: {"json":"message"}

You must now install the SixthSense log monitoring agent. For more informaiton, see Installing Log Monitoring agents.