# Flightglass collector for AWS A single, read-only Python script that reads configuration metadata from one AWS account and writes it to a JSON file (the "bundle"). You send that file to your Flightglass partner, who uploads it to produce an agentic AI readiness assessment. It is open source (Apache-2.0) so your security team can read exactly what it does before running it. Everything it reads is listed below. ## What it does and doesn't do **It does:** make read-only `List`, `Get` and `Describe` calls (plus IAM's `GenerateCredentialReport`, which asks IAM to refresh its built-in, read-only credential report), keep only the fields listed in the code for each call, and write one file on the machine where it runs. **It never:** - changes anything in your account (no `Put`, `Create`, `Update`, `Delete`, `Start` or `Tag` calls); - reads secret values, access keys, passwords or KMS key material; - downloads S3 objects or queries databases; - reads prompts or model outputs, or invokes a model; - sends anything anywhere. The file stays on your machine until you choose to share it. Where it can only count things, it keeps counts: for example the number of Secrets Manager secrets (never their names) and how many log groups have a retention period (never their names). To spot AI used outside Amazon Bedrock, it checks secret names and Lambda environment variable names (such as `OPENAI_API_KEY`) against a list of AI providers on your machine and keeps only how many refer to each provider. Names are never written to the file, and variable values are never kept (AWS returns them with the function list; the collector drops them). Credential report rows keep only whether each access key is active and when it was last rotated. For usage, it keeps totals only: Amazon Bedrock calls and tokens per model over the last 30 days (from CloudWatch metrics, never prompts, outputs or who made the calls), and the last three full months' spend on Amazon Bedrock and Amazon SageMaker. Spend on every other service is filtered out on your machine before the file is written. ## Running it **Quickest set-up (CloudFormation).** In AWS CloudShell, as someone who can create IAM roles, create a read-only `FlightglassCollector` role from the template in this folder, then run the collector through it: ```bash curl -fsSLO https:///collector/aws/flightglass-collector-role.json aws cloudformation deploy --template-file flightglass-collector-role.json \ --stack-name FlightglassCollector --capabilities CAPABILITY_NAMED_IAM python3 flightglass_collector.py --regions eu-west-2,eu-west-1 \ --role-arn arn:aws:iam:::role/FlightglassCollector ``` The role has exactly the permissions below (SecurityAudit, ViewOnlyAccess and the eight-action inline policy) and trusts only your own account: it gives Flightglass, and anyone outside your account, no access. Remove it afterwards with `aws cloudformation delete-stack --stack-name FlightglassCollector`. **Or set permissions by hand:** 1. **Permissions.** Use a role or user with the AWS managed policies [`SecurityAudit`](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/SecurityAudit.html) and [`ViewOnlyAccess`](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/ViewOnlyAccess.html), plus the small inline policy in [`iam-policy.json`](iam-policy.json). The inline policy adds eight read actions those managed policies don't include (checked against policy versions SecurityAudit v94 and ViewOnlyAccess v47). 2. **Open AWS CloudShell** in one of the regions you're assessing. Python 3 and boto3 are already installed there. 3. **Download and run** (your Flightglass partner's instructions page gives the exact command): ```bash python3 flightglass_collector.py --regions eu-west-2,eu-west-1 --output flightglass-bundle.json ``` It shows a summary of what it will read and asks you to confirm before reading anything. 4. **Download the file** from CloudShell (Actions › Download file › `flightglass-bundle.json`) and send it to your Flightglass partner. Run it once per account in scope. At the end it prints how many items it read, any it couldn't read (usually a missing permission or a feature that's switched off; the assessment reports these separately), the file's location and its SHA-256 hash, so you can confirm the file you send is the one it wrote. ### Options | Option | What it does | | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--regions` | Comma-separated regions to assess. Default: the current CloudShell region. | | `--output` | Output file. Default: `flightglass-bundle.json`. | | `--yes` | Skip the confirmation prompt (for scripted runs). | | `--dry-run` | List every call it would make, without calling AWS. Needs no credentials. | | `--redact-arns` | Replace resource names (roles, users, trails, budgets…) with short, stable hashes. Account IDs, AWS-managed policy names and policy contents are kept because the checks need them. | | `--profile` | Use a named AWS CLI profile instead of the current credentials. | | `--role-arn` | Read through this role (for example the `FlightglassCollector` role the CloudFormation template creates). It's assumed for this run only; the temporary credentials stay in memory. | **Cost:** AWS charges about $0.01 per Cost Explorer API request, and the collector makes three (cost allocation tags, anomaly monitors and AI spend), so a run costs about $0.03. CloudWatch charges about $0.01 per 1,000 metrics read with `GetMetricData`; the collector reads three per Bedrock model in use, so this is a fraction of a cent. Every other call it makes is free. Requirements: Python 3.9 or newer and boto3. No other packages. If your boto3 is older than the newest Amazon Bedrock APIs, those items are recorded as "too old, please update" and the run continues; update with `pip3 install --user --upgrade boto3`. ## Every call it makes "Each region" calls run once in every region you pass with `--regions`. Calls that depend on another (for example, describing each KMS key returned by `kms.list_keys`) only run for resources the first call found. | Call | Where | Permission | Why | | ---------------------------------------------------------- | -------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `sts.get_caller_identity` | Once | None needed | Which account and role the collector is running as. | | `iam.list_account_aliases` | Once | `iam:ListAccountAliases` | The account's friendly name, shown on the report (optional). | | `bedrock.list_foundation_models` | Each region | `bedrock:ListFoundationModels` | Whether Amazon Bedrock models are available in each region. | | `bedrock.get_model_invocation_logging_configuration` | Each region | `bedrock:GetModelInvocationLoggingConfiguration` | Whether model invocation logging is switched on, and where logs go. | | `bedrock.list_guardrails` | Each region | `bedrock:ListGuardrails` | Whether Bedrock Guardrails are defined. | | `bedrock.get_guardrail` | Each region, per guardrail | `bedrock:GetGuardrail` | Each guardrail's content filter types and strengths, to see whether prompt attacks are blocked. Topics, words, messages and other settings are dropped. | | `bedrock.list_enforced_guardrails_configuration` | Each region | `bedrock:ListEnforcedGuardrailsConfiguration` (inline policy) | Guardrails the account enforces on every model call, and which models they cover. | | `bedrock-agentcore-control.list_agent_runtimes` | Each region | `bedrock-agentcore:ListAgentRuntimes` | How many AgentCore agent runtimes already exist (count only). | | `bedrock-agentcore-control.list_gateways` | Each region | `bedrock-agentcore:ListGateways` | AgentCore gateways (how agents reach tools) and their sign-in type. | | `bedrock-agentcore-control.get_gateway` | Each region, per gateway | `bedrock-agentcore:GetGateway` | Each gateway's sign-in type, AgentCore Policy mode and number of interceptors. Nothing else is kept. | | `bedrock-agentcore-control.list_online_evaluation_configs` | Each region | `bedrock-agentcore:ListOnlineEvaluationConfigs` | Whether online evaluations score live agent traffic (status only). | | `bedrock-agent.list_agents` | Each region | `bedrock:ListAgents` | How many Bedrock Agents already exist (count only). | | `bedrock-agent.list_knowledge_bases` | Each region | `bedrock:ListKnowledgeBases` | How many Bedrock Knowledge Bases already exist (count only). | | `service-quotas.list_service_quotas` | Each region | `servicequotas:ListServiceQuotas` | Bedrock throughput quotas (ServiceCode bedrock). | | `iam.get_account_summary` | Once | `iam:GetAccountSummary` | Whether the root user has MFA (AccountMFAEnabled). | | `sso-admin.list_instances` | Each region | `sso:ListInstances` | Whether IAM Identity Center is set up. | | `iam.generate_credential_report` | Once | `iam:GenerateCredentialReport`, `iam:GetCredentialReport` | Age of active IAM user access keys. Only key status and rotation dates are kept; passwords, MFA serials and last-used details are dropped. | | `iam.get_credential_report` | Once | (as above) | Reads the report requested above. | | `accessanalyzer.list_analyzers` | Each region | `access-analyzer:ListAnalyzers` | Whether IAM Access Analyzer is active. | | `iam.get_account_authorization_details` | Once | `iam:GetAccountAuthorizationDetails` | Roles, their permission boundaries and attached policies, to count over-privileged roles. Users and groups are not kept. | | `glue.get_databases` | Each region | `glue:GetDatabases` | Whether a Glue Data Catalog is in use (database names only). | | `lakeformation.get_data_lake_settings` | Each region | `lakeformation:GetDataLakeSettings` | Whether Lake Formation data lake administrators are defined. | | `macie2.get_macie_session` | Each region | `macie2:GetMacieSession` | Whether Amazon Macie is enabled. | | `s3control.get_public_access_block` | Once | `s3:GetAccountPublicAccessBlock` | Account-level S3 Block Public Access settings. | | `secretsmanager.list_secrets` | Each region | `secretsmanager:ListSecrets` | How many secrets exist, and how many have names that refer to an AI provider such as OpenAI (counts per provider; names and values are never kept). | | `ec2.describe_vpc_endpoints` | Each region | `ec2:DescribeVpcEndpoints` | Whether private VPC endpoints exist for Bedrock (Bedrock endpoints only). | | `guardduty.list_detectors` | Each region | `guardduty:ListDetectors` | Whether a GuardDuty detector exists. | | `guardduty.get_detector` | Each region | `guardduty:GetDetector` | Whether each GuardDuty detector is enabled. | | `securityhub.describe_hub` | Each region | `securityhub:DescribeHub` | Whether AWS Security Hub is enabled. | | `kms.list_keys` | Each region | `kms:ListKeys` | KMS key IDs, to check customer-managed keys. | | `kms.describe_key` | Each region | `kms:DescribeKey` | Who manages each key (AWS or customer), its state and type. | | `kms.get_key_rotation_status` | Each region | `kms:GetKeyRotationStatus` | Whether automatic rotation is on, for customer-managed symmetric keys. | | `cloudtrail.describe_trails` | Once | `cloudtrail:DescribeTrails` | CloudTrail trails, including multi-region and organization trails. | | `cloudtrail.get_trail_status` | Once | `cloudtrail:GetTrailStatus` | Whether each trail is currently logging. Keyed by trail ARN. | | `cloudtrail.get_event_selectors` | Once | `cloudtrail:GetEventSelectors` | Which events each trail records, to find Bedrock data events. Keyed by trail ARN. | | `config.describe_configuration_recorder_status` | Each region | `config:DescribeConfigurationRecorderStatus` | Whether AWS Config is recording. | | `logs.describe_log_groups` | Each region | `logs:DescribeLogGroups` | How many log groups have a retention period set (counts only, no names). | | `xray.get_trace_segment_destination` | Each region | `xray:GetTraceSegmentDestination` | Whether traces go to CloudWatch Logs (Transaction Search) or X-Ray. | | `budgets.describe_budgets` | Once | `budgets:ViewBudget` | Whether AWS Budgets are defined (names and types only, no amounts). | | `ce.list_cost_allocation_tags` | Once | `ce:ListCostAllocationTags` | Which cost allocation tag keys are active. | | `ce.get_anomaly_monitors` | Once | `ce:GetAnomalyMonitors` | Whether Cost Anomaly Detection monitors exist. | | `cloudwatch.list_metrics` | Each region | `cloudwatch:ListMetrics` | Which Amazon Bedrock models have usage metrics (model IDs only). | | `cloudwatch.get_metric_data` | Each region | `cloudwatch:GetMetricData` | Bedrock calls, input tokens and output tokens per model over the last 30 days (totals only). | | `ce.get_cost_and_usage` | Once | `ce:GetCostAndUsage` | Last three full months' spend on Amazon Bedrock (including third-party models) and Amazon SageMaker. Other services are filtered out before the file is written. | | `lambda.list_functions` | Each region | `lambda:ListFunctions` | How many functions have environment variable names that refer to an AI provider (counts per provider; values and function names are dropped). | | `sagemaker.list_endpoints` | Each region | `sagemaker:ListEndpoints` | How many SageMaker endpoints are in service (count only). | | `organizations.describe_organization` | Once | `organizations:DescribeOrganization` | Whether the account belongs to an AWS Organization. | | `organizations.list_policies` | Once | `organizations:ListPolicies` | Service control policies and tag policies in the organization. | | `organizations.describe_policy` | Once | `organizations:DescribePolicy` | The content of each policy, to look for Bedrock and region restrictions. | | `organizations.list_targets_for_policy` | Once | `organizations:ListTargetsForPolicy` | Where each policy is attached (root, OUs or accounts). | | `organizations.describe_effective_policy` | Once | `organizations:DescribeEffectivePolicy` | The Amazon Bedrock policy in effect for this account, to see which guardrails are enforced organization-wide. Keyed by policy type (BEDROCK_POLICY). Works from member accounts. | | `controltower.list_landing_zones` | Each region | `controltower:ListLandingZones` | Whether an AWS Control Tower landing zone exists. | ## Data handling The bundle holds configuration metadata only: account and resource IDs (needed as evidence in the report), settings and counts. Use `--redact-arns` if you'd rather not share resource names. Flightglass stores uploaded bundles in the UK (London) and deletes raw bundles 90 days after upload. How it stores, protects and deletes them, and how to report a vulnerability, is in [SECURITY.md](../../docs/SECURITY.md). ## Development ```bash python3 -m venv collector/aws/.venv collector/aws/.venv/bin/pip install boto3 jsonschema collector/aws/.venv/bin/python -m unittest discover -s collector/aws/tests -v ``` Tests run against a fake AWS that checks every request and response against boto3's own service definitions, so no AWS account or network is needed. The output is validated against the published bundle schema (`packages/schema/bundle.schema.json`). ## Licence Apache License 2.0. See [LICENSE](LICENSE).