Use case, Input forward, Output aws s3
==============================================================================

Make sure your EC2 have AWS S3 IAM Role attached.

**Run fluent-bit service container**::

    # Setup variables
    repo_name="fluent/fluent-bit" # https://hub.docker.com/r/fluent/fluent-bit/
    container_name="fluent-bit-deamon-set"
    s3_bucket="ociso-for-everything"

    # output example3 - AWS S3
    # input=forward https://docs.fluentbit.io/manual/pipeline/inputs/forward
    # output=standard-out https://docs.fluentbit.io/manual/pipeline/outputs/s3
    #
    # note that fluent-bit don't write logs to s3 bucket immediately when receives it
    # it store it in buffer and write to it when reaching the upload_chunk_size (minimal 5M)
    # or upload_timeout (minimal 1M)
    sudo docker run --rm --name "${container_name}" -p 127.0.0.1:24224:24224 ${repo_name} /fluent-bit/bin/fluent-bit -i forward -o s3 -p region=us-east-1 -p bucket=${s3_bucket} -p total_file_size=1M -p upload_timeout=1m -f 1

**Run app container, generate some logs**::

    # Setup variables
    repo_name="ubuntu" # https://hub.docker.com/_/ubuntu

    # Since fluent-bit s3 only write to S3 when
    # reaching minimal upload_timeout, which is 1m in this case
    # reaching upload_chunk_size, which is 5M in this case,
    # in order to see the log immediately, I create a simple python script
    # generate_big_file.py
    # that creating a 5MB text file, and then log the content to it.

    # copy the python script to generate_big_file.py
    vi generate_big_file.py

    # create big file: big-file.txt
    python generate_big_file.py

    # send some log
    sudo cat big-file.txt | sudo docker run --log-driver=fluentd -t ubuntu echo

    # or just send some small log and wait for 1 min
    sudo docker run --log-driver=fluentd -t ubuntu echo Hello World

    # go to s3 console to view the log
    https://s3.console.aws.amazon.com/s3/buckets/ociso-for-everything?region=us-east-1&prefix=fluent-bit-logs/

**What if fluent-bit container crushed, Do we lose the data in buffer**?

If the fluent-bit container doesn't bind to any volume, YES your lose the data in buffer. That's why from the official doc https://docs.fluentbit.io/manual/pipeline/outputs/s3, it recommends to send to s3 more frequently.

From the AWS document https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_data_volumes.html, all of ECS, Fargate, EKS supports using data volume. So we should have no problem.