Containerized Embedded Development: The Right Tools for Reproducible Workflows

Missed Part 3 of the “Practical Embedded Development, Containerized” series? Read about Bridging the Three Worlds in Containerized Embedded Develoment first.

You’re sold on containerizing embedded development—but the wrong tools mean fighting daemon overhead, security risks, or cloud development environments that can’t reach your JTAG debugger or USB hardware. As this series has shown, containers eliminate "works on my machine" problems by packaging toolchains and debugger setups into reproducible, portable environments.

Now it’s time to pick the right tools for containerized embedded workflows. This guide covers:

• ✅ Podman vs. Docker: Why Podman’s daemonless design wins for security and performance—and where Docker’s polished UI and multi-platform builds still excel.

• ✅ VSCode DevContainers: How the open DevContainer Spec turns containers into standardized IDEs—with direct hardware access.

• ✅ GitHub Codespaces: When cloud development environments work (and where they fail for embedded development).

• ✅ Other Tools: Beyond Docker/Podman for legacy toolchains, on-premises CI with hardware access, and Kubernetes-native workflows.

No vendor lock-in. No hand-wavy advice. Just industry-standard tools for embedded engineers.

OCI and DevContainer Standards: The Foundation

Before comparing tools, let’s ground ourselves in the standards that make them interchangeable. These aren’t theoretical—they’re what let you build once, run anywhere, without rewriting configs when tools evolve.

OCI Standard: The USB of Containers

The Open Container Initiative (OCI) standardizes containers. Just as USB lets any device plug into any computer—regardless of brand or OS—OCI ensures any container runs on any compliant tool (Podman, Docker, Kubernetes, etc.). It defines three critical specs:

• Runtime Spec: How containers execute: process isolation, filesystem bundles—the “connector shape”.

• Image Spec: How images are structured, built, and shared—the “data transfer protocol”.

• Distribution Spec: How container images are stored and pulled from registries—the "USB hub".

Why it matters for embedded:

• Tool agnosticism: Build with Podman, debug with Docker, deploy with Kubernetes—same image, zero changes.

• Future-proofing: Your 2020 Dockerfile still works today (like USB 2.0 in a USB-C port with an adapter).

• Ecosystem leverage: CI systems, registries, and orchestrators all “speak OCI”.

Example: A Podman-built image runs unchanged in Docker, containerd, or AWS ECS.

DevContainer Spec: IKEA Instructions for Your Dev Environment

While OCI standardizes containers, the DevContainer Specification standardizes development environments. Just as IKEA’s flat-pack designs let you assemble the same bookshelf in any home, this spec lets you replicate a full workspace (toolchain, debuggers, IDE settings) from a single devcontainer.json file.

Key Components:

• Configuration format (.devcontainer.json): Blueprint for base image, runtime args, extensions, and hardware access.

• Features: Modular components (e.g., common-utils, docker-in-docker) to avoid reinventing setups.

Why it matters for teams:

• Shareable configs: One file (devcontainer.json) replicates your entire environment.

• IDE-agnostic: Officially supported by VSCode and GitHub Codespaces; adopted by other IDEs (e.g., JetBrains IDEs, Eclipse).

• Hardware-aware: USB/JTAG passthrough works if your host OS permits it.

Example: Minimal embedded setup with a custom Dockerfile control:

{
  "name": "Custom Embedded DevContainer",
  "build": {
    "dockerfile": "./Dockerfile"
  },
  "runArgs": ["--device=/dev/ttyACM0"],
  "customizations": {
    "vscode": {
      "extensions": ["ms-vscode.cpptools"]
    }
  }
}

---

Standards turn containers from "works on my machine" to "works on every machine"—with no lock-in.

These standards enable an entire ecosystem. Here’s how the pieces fit together:

Podman vs. Docker: Architectural Trade-offs

Both Podman and Docker implement OCI standards, but their underlying designs create key differences:

Podman’s Strength

Daemonless architecture means no persistent dockerd process to manage or debug. This translates to:

• Faster container startup times

• Cleaner resource isolation

• No single point of failure behind containers

Security-first design with rootless operation by default. For embedded work involving proprietary toolchains and hardware access, this reduces:

• Attack surface area

• Compliance risks in regulated environments

Fully open-source implementation gives full auditability.

Docker's Strengths

Polished user experience through Docker Desktop:

• Intuitive GUI for container management

• Built-in dashboard for monitoring resources

• Easier onboarding for teams new to containers

Mature ecosystem with:

• buildx for well-documented multi-platform builds

• Extensive plugin support (volume drivers, logging, etc.)

⚠️ Podman’s VM Layer on Mac/Windows

• Set VM resources upfront:

    podman machine set --cpus=4 --memory=4096 --disk-size=20
  

• USB/JTAG passthrough:

  • macOS: Manual attachment needed.

  • Windows: USB/IP forwarding needed, which may add latency.

  • Test early—some debuggers misbehave through the VM.

The Bottom Line

Choose Podman if you prioritize:

• ✅ Security and compliance

• ✅ Kubernetes-aligned architecture

• ✅ Linux-native workflows (best experience)

• ✅ Open-source purity

Choose Docker if you need:

• ✅ Easiest team onboarding

• ✅ Multi-platform build maturity

• ✅ Windows compatibility (native device access)

• ✅ Proprietary tooling integrations

For most embedded teams—especially on Linux—Podman's advantages outweigh Docker's convenience. However, Windows users may prefer Docker for native device access.

---

But selecting a runtime is only half the battle. The real challenge? Ensuring every engineer—from the CI pipeline to the hardware bench—uses the same environment, with the same tools, and the same access to devices. That’s where VSCode DevContainers bridges the gap: turning containers into reproducible workspaces that travel with your project.

VSCode DevContainers: IDE as Code

VSCode DevContainers turn containers into reproducible workspaces by encoding your environment—IDE settings, extensions, and hardware access—in version-controlled configs. The only setup required: cloning the repo.

Why It’s a Game-Changer

1. Vendor-Neutral Foundation
   It works identically across systems via the open DevContainer Specification.

2. IDE as Code
   Your .devcontainer/devcontainer.json defines:

   • Toolchain versions (compilers, debuggers)

   • IDE extensions (C/C++ tools, Cortex-Debug)

   • Runtime arguments (device passthrough, privileges)

3. Hardware-Awareness
   Devices expose to the container via your host’s Docker/Podman. You configure all hardware access in devcontainer.json just as you would for a standalone container.

Example config:

{
  "image": "ghcr.io/your-org/embedded-dev:latest",
  "runArgs": ["--device=/dev/ttyUSB0"],  // Direct USB access
  "customizations": {
    "vscode": {
      "extensions": [
        "ms-vscode.cpptools"            // C/C++ support
      ],
      "settings": {
        "terminal.integrated.shell.linux": "/usr/bin/bash"
      }
    }
  },
  "features": {
    "ghcr.io/devcontainers/features/common-utils:2": {}  // git, curl, etc.
  }
}

Setup Steps for VSCode DevContainers

• Setup requirement: Install the VSCode DevContainers extension. It works with Docker out-of-the-box; for Podman, configure the Docker executable path in VSCode settings to point to podman.

• Workflow: When opening a project with DevContainers configured, use the DevContainers: Reopen in Container command to start the containerized environment.

• User experience: Once running in a container, VSCode behaves identically to local development or remote SSH—just with your environment fully containerized.

---

DevContainers solve local standardization—but what about the cloud?

GitHub Codespaces: Cloud Portability (With Limits)

GitHub Codespaces extends DevContainers to the cloud, offering browser-accessible environments—ideal for collaboration but useless for hardware debugging.

The Embedded Limitation

❌ No hardware access.

Codespaces runs in a remote VM without local device connectivity. This means that your embedded hardware (JTAG debuggers, USB devices, serial ports) remains completely unavailable. Use Codespaces only for cloud-native tasks (e.g., code reviews, documentation, build testing).

Where It Excels

✅ Cloud-native tasks: Spin up identical environments for code reviews, documentation, building, and simulation.

✅ Online collaboration: Shared environments for pair programming or onboarding.

✅Zero local setup: Develop from any machine with just a browser (or VSCode desktop).

✅ Preconfigured environments: Uses the same devcontainer.json as local DevContainers.

Availability & Usage

• Free for personal accounts (with usage limits) on GitHub-hosted repositories.

• Enterprise/managed accounts require paid access.

• Best practice: Reserve Codespaces for occasional online collaboration or development on the go (when hardware debugging isn’t needed). Combined with GitHub Actions, it’s ideal for CI-based build testing and simulations.

---

Choosing between local and cloud DevContainers? Here’s how they stack up for embedded development:

The Broader Ecosystem

While Docker/Podman + DevContainers cover most use cases, the container ecosystem is much broader and offers tools that you might encounter.

Container Runtimes

Container Engines as Docker and Podman rely on Container Runtimes like containerd (or directly on runc/crun) under the hood. You can use containerd directly for minimal overhead, but expect no dev-friendly features—it’s a plumbing layer, not a replacement. nerdctl is a Docker-like CLI for containerd, useful if you’re in a Kubernetes environment and miss docker build.

For experimental setups, lilipod (a minimal and limited container runtime in Rust) and youki (an OCI runtime also in Rust) offer lighter-weight options, but neither is production-ready for embedded work yet.

Alternative Environment

distrobox lets you run full Linux distributions (e.g., Ubuntu 18.04) as containers—ideal for legacy toolchains or proprietary EDA tools without VMs.

DevContainers in Other IDEs

The DevContainer Spec isn’t VSCode-exclusive. JetBrains IDEs and Visual Studio support it officially. CodeSandbox provides a cloud development environment similar to GitHub Codespaces.

Devfiles is a vendor-neutral alternative to DevContainers. Reproducible environments are defined in a devfile.yaml, which is optimized for cloud/remote workspaces and supported by Eclipse Che and OpenShift DevSpaces. Devfiles are less mature for embedded work due to limited hardware passthrough. Worth monitoring if you’re in a Kubernetes-heavy ecosystem.

CI with Hardware

For on-premises CI with JTAG/USB access, use:

• Kubernetes + device plugins (complex but powerful).

• GitHub Actions + self-hosted runners (simpler, if your runners have USB passthrough).

• Helm to package/deploy containerized toolchains to your own infrastructure.

Further Reading & Experiments

To go deeper, here are practical resources for containerized embedded development:

Bookmark Suggestions

Podman:

• Podman Rootless Guide Critical for security-compliant embedded development.

DevContainers Community:

• DevContainer Specification The "IKEA instructions" for your development environment—study this before customizing.

• Official DevContainer Templates Prebuilt configurations for common languages/tools.

• Official DevContainer Resources All repositories from the DevContainer Community.

Embedded-Specific DevContainers:

• Embedded Container Resources A curated collection of embedded-specific DevContainers.

• ARM Cortex-M DevContainer A DevContainer for developing and debugging on ARM Cortex-M.

• Philips AMP DevContainer DevContainers for embedded C++ and Rust development.

Experiments to Try

1. Legacy Toolchain: Containerize GCC 5.4 in distrobox to replace a VM

2. Debugger Test: Compare Podman vs. Docker with your JTAG debugger.

3. CI Pipeline: Use GitHub Actions + Codespaces to test builds without hardware, and then with a self-hosted runner with USB passthrough.

---

The container ecosystem is ready for embedded work today. Start with Docker/Podman + DevContainers, then explore advanced tools as needed. The frontier? Standardizing hardware access across runtimes and clouds.