Work record / Vue.ai / 2023-now
Multi-cloud foundations
One library over AWS, Azure, GCP, OCI and bare metal, and the infrastructure work built around it.
- multi-cloud
- bare metal
- compatibility
Context
I maintain and extend an internal multi-cloud library that abstracts operations across AWS, Azure, GCP, OCI, and bare metal, so callers don’t need to know cloud provider sdk specifics. More than eight services depend on it, making backward compatibility a constraint on every change.
Alongside this, I added bare-metal support for secrets and MinIO-backed storage in the same multi-cloud package, built account specific cloud infrastructure deployments that can be added or removed on-demand through a DAG-based template and handled partial database migrations for an internal service.
Approach
Consumer friendly abstraction. The library exposes a shared interface for common operations across providers, covering streams, blob storage, secrets, wide-column and key-value databases, and queues. Keeping these operations behind a consistent interface lets services work across providers without having to account for their internal implementations, speeding up integrations.
Do not suppress native capabilities. A common abstraction shouldn’t mean losing useful provider-specific features. In OCI, updating a secret creates a new version, and unlike other cloud providers, OCI imposes an upper limit on those versions. I tracked down provider sdk specific options (some disabled by default) to avoid unnecessary version bumps and prevent race conditions. I applied the same principle in data pipelines by enabling Azure append blobs rather than restricting storage operations to what every provider supports (which is COW (copy on write)).
Ensure the critical data consistency, and backfill the lost parts. A database used by an open-source tool was left in an inconsistent state, and I volunteered in recovering it. I first identified the mission-critical tables and verified their consistency. Non-critical data, such as job counters, could be reconstructed from traces found elsewhere in other services. To understand the database schema, I checked out the tool’s exact release tag and used ER diagarams & sourcegraph (R.I.P.) to help me trace how the code mapped to it. I then compared the data, restored the database in a client environment, and brought the affected services back to a healthy state.
Boundaries
- The shared interface covers common operations across providers.
- Provider-specific capabilities, such as Azure append blobs, are utilized where supported.