Configuring Loki to use AWS S3 Storage in DKP

Follow the instructions on this page to configure Loki to use AWS S3 Storage in DKP

Configuring Loki

1. Execute the following command to get the namespace of your workspace:

   CODE

   dkp get workspaces

2. Set the WORKSPACE_NAMESPACE variable to the namespace copied in the previous step:

   CODE

   export WORKSPACE_NAMESPACE=<WORKSPACE_NAMESPACE>

3. Create a secret containing the static AWS S3 credentials. The secret is then mounted into each of the Grafana Loki pods as environment variables. 

   CODE

   kubectl create secret generic dkp-aws-s3-creds -n${WORKSPACE_NAMESPACE} \
   --from-literal=AWS_ACCESS_KEY_ID=<key id> \
   --from-literal=AWS_SECRET_ACCESS_KEY=<secret key>

4. Create a config overrides ConfigMap to update the storage configuration:
   NOTE: This could also be added to the installer configuration if you are configuring Grafana Loki on the Management Cluster

   CODE

   cat <<EOF | kubectl apply -f -
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: grafana-loki-overrides
     namespace: ${WORKSPACE_NAMESPACE}
   data:
     values.yaml: |
       loki:
         annotations:
           secret.reloader.stakater.com/reload: dkp-aws-s3-creds
         structuredConfig:
           storage_config:
             aws:
               s3: s3://<region>/<bucket name>
       ingester:
         extraEnvFrom:
           - secretRef:
               name: dkp-aws-s3-creds 
       querier:
         extraEnvFrom:
           - secretRef:
               name: dkp-aws-s3-creds 
       queryFrontend:
         extraEnvFrom:
           - secretRef:
               name: dkp-aws-s3-creds 
       compactor:
         extraEnvFrom:
           - secretRef:
               name: dkp-aws-s3-creds  
       ruler:
         extraEnvFrom:
           - secretRef:
               name: dkp-aws-s3-creds 
       distributor:
         extraEnvFrom:
           - secretRef:
               name: dkp-aws-s3-creds
   EOF

5. Update the grafana-loki AppDeployment to apply the configuration override.

   NOTE: If you use the Kommander CLI installation configuration file, you don’t need this step

   CODE

   cat << EOF | kubectl -n ${WORKSPACE_NAMESPACE} patch appdeployment grafana-loki --type="merge" --patch-file=/dev/stdin
   spec:
     configOverrides:
       name: grafana-loki-overrides
   EOF