Daily Backup Reports by Group

Introduction

We are excited to announce a significant enhancement to the base2tools reporting system. The daily backup reports have been completely refactored to provide group-level reporting instead of per-contract reporting, along with the addition of automated email distribution capabilities through AWS Simple Email Service (SES). This update streamlines backup monitoring for organizations managing multiple contracts by consolidating reports at the group level, making it easier to get a comprehensive view of backup status across related contracts.

Features

Group-Level Daily Backup Reports

The reporting system has been fundamentally restructured to aggregate backup reports at the group level rather than generating individual reports for each contract. This provides several key benefits:

• Consolidated View: All contracts within a group are now reported together, providing a unified view of backup status across related contracts
• Improved Organization: Reports are now organized by group, making it easier to manage and monitor backups for organizations with multiple contracts
• Historical Data Storage: Daily reports are now archived with a date-based folder structure (year/month/day) for better historical tracking and compliance

Automated Email Distribution via AWS SES

A new email distribution system has been implemented to automatically send daily backup reports to designated recipients:

• AWS SES Integration: Leverages AWS Simple Email Service for reliable email delivery
• Configurable Recipients: Support for multiple email recipients through environment configuration
• CSV Attachments: Reports are automatically attached as CSV files for easy analysis and record-keeping
• Email Validation: Built-in email validation ensures only properly formatted email addresses are used

Enhanced Infrastructure Support

The solution includes several infrastructure improvements:

• New DynamoDB Table: Added support for a ContractsGroupsTable to manage group-contract relationships
• SES Configuration: New parameters for configuring the SES domain and recipient email addresses
• Queue-Based Processing: Updated queue populator to handle group-based message distribution

Examples

Configuration Parameters

When deploying the updated reporting system, you’ll need to configure the following new parameters:

# Development Environment
ContractsGroupsTable: "gateway-dev-contracts-groups"
SESDomain: "tools.aws.base2.services"
SESToEmails: "team@example.com,manager@example.com"

# Production Environment  
ContractsGroupsTable: "gateway-prod-contracts-groups"
SESDomain: "tools.aws.base2.services"
SESToEmails: "ops-team@example.com"

Group Structure in DynamoDB

The new ContractsGroupsTable should be structured with the following schema:

{
  "group": "finance-team",
  "contract": "contract-001"
}

Multiple contracts can belong to the same group:

[
  {
    "group": "finance-team",
    "contract": "contract-001"
  },
  {
    "group": "finance-team", 
    "contract": "contract-002"
  },
  {
    "group": "engineering-team",
    "contract": "contract-003"
  }
]

Report Output Structure

Daily backup reports are now saved in S3 with the following structure:

backups/
├── finance-team/
│   ├── daily_backup_report.csv
│   └── 2025/
│       └── 4/
│           └── 11/
│               └── daily_backup_report.csv
└── engineering-team/
    ├── daily_backup_report.csv
    └── 2025/
        └── 4/
            └── 11/
                └── daily_backup_report.csv

Email Report Format

Recipients will receive emails with the following characteristics:

• Subject: “Daily Backup Report for [Group Name]”
• From Address: noreply@[your-ses-domain]
• Attachment: daily_backup_report.csv containing all backup data for contracts in the group
• Body: Simple notification text indicating the attached report

Lambda Function Updates

The queue populator Lambda function now processes groups instead of individual contracts:

# Old approach (per contract)
messages.append({
    'Id': str(uuid.uuid4()),
    'MessageBody': json.dumps({
        'contract': contract
    })
})

# New approach (by group)
messages.append({
    'Id': str(uuid.uuid4()),
    'MessageBody': json.dumps({
        'group': group,
        'contracts': get_contracts_for_group(group)
    })
})

Conclusion

This release represents a significant improvement in how daily backup reports are generated and distributed within the base2tools reporting system. By moving to group-level reporting and adding automated email distribution, organizations can now more efficiently monitor and manage their backup operations across multiple contracts. The addition of historical data archiving also provides better compliance and audit capabilities.

To implement these changes, ensure you:

1. Update your CloudFormation/SAM templates with the new parameters
2. Create and populate the ContractsGroupsTable with your group-contract mappings
3. Verify your SES domain is properly configured for sending emails
4. Update recipient email addresses in your environment configuration

For any questions or support regarding this update, please contact the base2Services team.