AWS S3 — IAM role and bucket setup for the engine

The engine writes traffic-log batches directly to your S3 bucket using the IAM role attached to its EC2 instance. Your AWS account owns the bucket and the role; Enforza never holds AWS credentials for you. This article walks through the four things you set up on your side before creating the AWS S3 sink in the console.

Note

If the role or bucket isn’t right, the engine will fail to write and surface a per-sink error chip on the firewall row. The configuration in the console is just a pointer; AWS does the actual access check at write time.

1. Create (or pick) the S3 bucket

1. Pick a region — typically the same region the engine VM runs in to avoid cross-region transfer cost.
2. Create the bucket, e.g. acme-enforza-archive.
3. Block all public access (default).
4. Enable default encryption — SSE-S3 (AES256) is the simplest; SSE-KMS works if you grant the engine role kms:Encrypt/GenerateDataKey on the key.
5. (Recommended) Lifecycle rule to transition objects to S3 Glacier Instant Retrieval after 30 days, expire after your retention policy.

2. Identify the engine VM’s IAM role

The engine runs as a systemd service on an EC2 instance. The instance has an IAM instance profile; the role inside that profile is what the engine uses for AWS calls. Find it in the EC2 console under the instance’s Security → IAM role. Note the role’s ARN, e.g. arn:aws:iam::123456789012:role/enforza-engine.

Heads-up

If the instance has no role attached, create one and attach it (instance can stay running; attach via “Modify IAM role”). Without an instance role the engine has no AWS identity at all and the S3 sink can never work.

3. Grant the role s3:PutObject on the bucket prefix

Add an inline policy to the engine’s IAM role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "EnforzaEngineWriteLogs",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Resource": "arn:aws:s3:::acme-enforza-archive/fw-logs/*"
    }
  ]
}

Scope the Resource to the prefix you’ll configure in the sink editor. Don’t grant s3:ListBucket or s3:GetObject — the engine only writes.

If the bucket uses SSE-KMS

{
  "Sid": "EnforzaEngineUseKmsKey",
  "Effect": "Allow",
  "Action": [
    "kms:Encrypt",
    "kms:GenerateDataKey"
  ],
  "Resource": "arn:aws:kms:eu-west-2:123456789012:key/<kms-key-uuid>"
}

4. Cross-account buckets (optional)

If the bucket lives in a different AWS account than the engine, add a bucket policy on the target account that allows the engine role to write:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "EnforzaEngineCrossAccountWrite",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::123456789012:role/enforza-engine" },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::acme-enforza-archive/fw-logs/*"
    }
  ]
}

5. Create the sink in the console

On the Log Export page, click “New export” → AWS S3. Fill in:

• Bucket — exactly as in step 1.
• Region — the bucket’s region.
• Key prefix — defaults to fw-logs/{engine_id}/{yyyy}/{mm}/{dd}/. The placeholders are expanded by the engine; the trailing slash is required for a “directory” feel in the AWS console.
• Format — gzipped-ndjson recommended; ~5–10× smaller objects than uncompressed.
• Batching — flush every 60 s or 5 MB. Tune up for archival, down for near-real-time.

Advanced — copy-paste setup script

Heads-up

For experienced operators with sufficient AWS permissions (s3:CreateBucket, iam:CreatePolicy, iam:AttachRolePolicy, ec2:ModifyInstanceAttribute). Review the script before running. The script is idempotent — safe to re-run; it skips the bucket / policy / attachment if they already exist. Defaults match the Console “New export → AWS S3” form so you can paste the resulting bucket / prefix straight into the sink.

Fill in these variables at the top, then paste the whole block into a shell — or download the file and run ./enforza-s3-sink-setup.sh.

#!/usr/bin/env bash
# ── FILL IN ─────────────────────────────────────────────────────
BUCKET="acme-enforza-archive"          # globally-unique S3 bucket name
REGION="eu-west-2"                     # bucket region
PREFIX="fw-logs/"                      # key prefix the engine writes under
ENGINE_INSTANCE_ID="i-0123456789abcdef0"   # engine VM's EC2 instance id
                                           #   ↳ found in the EC2 console, or:
                                           #   aws ec2 describe-instances \
                                           #     --filters Name=tag:Name,Values=enforza-* \
                                           #     --query 'Reservations[].Instances[].InstanceId' --output text
# ────────────────────────────────────────────────────────────────

set -euo pipefail

# Preflight 1: confirm we have credentials AND they target the right account.
# Each AWS CLI call uses these credentials — get them wrong now and every
# subsequent step fails with AccessDenied, not because the script is broken
# but because nothing's authenticated against the target account.
echo "▶ Verifying AWS credentials…"
if ! CALLER=$(aws sts get-caller-identity --query Arn --output text 2>&1); then
  echo "  ✗ No AWS credentials in this shell. Run your usual auth flow"
  echo "    (aws configure, aws sso login, assume-role, etc.) and retry."
  exit 1
fi
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
echo "  using:    $CALLER"
echo "  account:  $ACCOUNT"
read -r -p "  ↳ Continue against this account? [y/N] " CONFIRM
[[ "$CONFIRM" =~ ^[Yy]$ ]] || { echo "  aborted."; exit 1; }

# Preflight 2: resolve the IAM role attached to the engine instance.
# We ask for the instance id (visible at a glance in the EC2 console)
# rather than the role name (which the operator may not even know).
echo "▶ Resolving the engine's IAM role from ${ENGINE_INSTANCE_ID}…"
PROFILE_ARN=$(aws ec2 describe-instances \
  --region "$REGION" --instance-ids "$ENGINE_INSTANCE_ID" \
  --query 'Reservations[0].Instances[0].IamInstanceProfile.Arn' \
  --output text 2>/dev/null || true)
if [[ -z "$PROFILE_ARN" || "$PROFILE_ARN" == "None" ]]; then
  echo "  ℹ Instance $ENGINE_INSTANCE_ID has no IAM instance profile attached."
  echo "    Without one, the engine has no AWS identity and the S3 sink can't work."
  read -r -p "  ↳ Create a role + profile now and attach it? [y/N] " CONFIRM
  [[ "$CONFIRM" =~ ^[Yy]$ ]] || { echo "  aborted."; exit 1; }

  NEW_ROLE_NAME="enforza-engine-${ENGINE_INSTANCE_ID#i-}"
  NEW_PROFILE_NAME="$NEW_ROLE_NAME"
  echo "  creating role + profile: $NEW_ROLE_NAME"
  cat > /tmp/enforza-trust.json <<'TRUST'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"ec2.amazonaws.com"},"Action":"sts:AssumeRole"}]}
TRUST
  aws iam create-role \
    --role-name "$NEW_ROLE_NAME" \
    --assume-role-policy-document file:///tmp/enforza-trust.json \
    --description "Enforza engine instance role (created by S3 sink setup script)" >/dev/null
  aws iam create-instance-profile \
    --instance-profile-name "$NEW_PROFILE_NAME" >/dev/null
  aws iam add-role-to-instance-profile \
    --instance-profile-name "$NEW_PROFILE_NAME" \
    --role-name "$NEW_ROLE_NAME"
  # IAM is eventually-consistent; the associate call below 400s with
  # NoSuchEntity if we don't give it a beat to propagate.
  echo "  waiting 10s for IAM consistency before attach…"
  sleep 10
  aws ec2 associate-iam-instance-profile \
    --region "$REGION" \
    --instance-id "$ENGINE_INSTANCE_ID" \
    --iam-instance-profile Name="$NEW_PROFILE_NAME" >/dev/null
  PROFILE_NAME="$NEW_PROFILE_NAME"
  ENGINE_ROLE_NAME="$NEW_ROLE_NAME"
  echo "  ✓ profile $NEW_PROFILE_NAME attached to $ENGINE_INSTANCE_ID"
else
  # Instance profile ARN: arn:aws:iam::123:instance-profile/<profile-name>.
  # The profile and role usually share a name but not always — resolve the
  # actual role(s) via the profile rather than guessing from the ARN.
  PROFILE_NAME=$(printf '%s' "$PROFILE_ARN" | awk -F'/' '{print $NF}')
  ENGINE_ROLE_NAME=$(aws iam get-instance-profile \
    --instance-profile-name "$PROFILE_NAME" \
    --query 'InstanceProfile.Roles[0].RoleName' --output text 2>/dev/null || true)
  if [[ -z "$ENGINE_ROLE_NAME" || "$ENGINE_ROLE_NAME" == "None" ]]; then
    echo "  ✗ Instance profile $PROFILE_NAME has no role attached. Aborting."
    exit 1
  fi
fi
echo "  profile:  $PROFILE_NAME"
echo "  role:     $ENGINE_ROLE_NAME"

echo "▶ Creating bucket s3://$BUCKET in $REGION (if missing)…"
if aws s3api head-bucket --bucket "$BUCKET" 2>/dev/null; then
  echo "  bucket already exists, skipping create"
else
  if [[ "$REGION" == "us-east-1" ]]; then
    aws s3api create-bucket --bucket "$BUCKET" --region "$REGION"
  else
    aws s3api create-bucket --bucket "$BUCKET" --region "$REGION" \
      --create-bucket-configuration "LocationConstraint=$REGION"
  fi
fi

echo "▶ Locking down public access + enabling versioning + default encryption…"
aws s3api put-public-access-block --bucket "$BUCKET" \
  --public-access-block-configuration \
  "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
aws s3api put-bucket-versioning --bucket "$BUCKET" \
  --versioning-configuration Status=Enabled
aws s3api put-bucket-encryption --bucket "$BUCKET" \
  --server-side-encryption-configuration \
  '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"}}]}'

echo "▶ Adding 30-day-to-Glacier lifecycle on ${PREFIX}…"
cat > /tmp/lifecycle.json <<EOF
{
  "Rules": [{
    "ID": "enforza-archive-transition",
    "Status": "Enabled",
    "Filter": { "Prefix": "$PREFIX" },
    "Transitions": [{ "Days": 30, "StorageClass": "GLACIER_IR" }]
  }]
}
EOF
aws s3api put-bucket-lifecycle-configuration --bucket "$BUCKET" \
  --lifecycle-configuration file:///tmp/lifecycle.json

echo "▶ Creating + attaching IAM policy on role ${ENGINE_ROLE_NAME}…"
POLICY_NAME="enforza-write-${BUCKET//[^A-Za-z0-9]/-}"
cat > /tmp/engine-write-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "EnforzaEngineWriteLogs",
    "Effect": "Allow",
    "Action": ["s3:PutObject", "s3:PutObjectAcl"],
    "Resource": "arn:aws:s3:::$BUCKET/$PREFIX*"
  }]
}
EOF
aws iam put-role-policy --role-name "$ENGINE_ROLE_NAME" \
  --policy-name "$POLICY_NAME" \
  --policy-document file:///tmp/engine-write-policy.json
echo "  policy $POLICY_NAME attached"

echo
echo "✔ Done. In the Enforza console → New export → AWS S3:"
echo "    Bucket:        $BUCKET"
echo "    Region:        $REGION"
echo "    Key prefix:    $PREFIX{engine_id}/{yyyy}/{mm}/{dd}/"
echo "    Format:        gzipped-ndjson"

Troubleshooting

• “AccessDenied” in the engine logs — the instance role doesn’t have s3:PutObject on the resource ARN. Re-check the inline policy resource pattern matches your prefix.
• “NoSuchBucket” — bucket name typo, or the bucket is in a different region than configured. Region must match exactly.
• “InvalidArgument: x-amz-server-side-encryption” — bucket requires KMS; either switch to AES256 or grant the role KMS permissions (see step 3).
• Objects appear but are 0 bytes — the engine’s per-sink buffer is empty. Wait for the next batch interval; if persistent, the engine writer is misconfigured upstream.