Tangram Bootstrap

This package provides the preliminary components required to bootstrap the Tangram ecosystem on Linux and macOS.

TL;DR Run make -j4 in this directory. On macOS, use make -j4 all_darwin to build both architectures.

Use the provided Makefile to produce these components ahead of running Tangram. Basic usage:

• make - Build all host platform components.
• make list - Enumerate the host platform components.
• make clean - Remove all build artifacts but retain downloaded sources.
• make clean_all - Remove all build artifacts AND sources.

Completed components and archives appear in dist/, with checksums in dist/SHASUMS256.txt. Only supported, completed components are archived.

Archives omit AppleDouble sidecars, Finder metadata, extended attributes, ACLs, file flags, and build stamps. Packaging uses the system tar on macOS or GNU tar on Linux.

On macOS, the list_all_platforms target enumerates every available component/platform combination. The makefile can optionally build the Linux targets as well using Docker Desktop. Use make all_platforms to build every available target.

See Prerequisites about the required host environment, Components about the included software, and Usage about additional provided targets.

Definitions

• artifact - Anything produced as a result of running a make target.
• component - A component the bootstrap package expects to provide, such as dash or toolchain.
• platform - Either x86_64_linux, aarch64_linux, x86_64_darwin, or aarch64_darwin.
• target - An action supported by this makefile. These can be "phony" (clean, toolchain) or refer to an actual output file: $SOURCEDIR/dash-0.5.13.5.tar.gz.

Prerequisites

This makefile is intended to be as portable as possible. However, it must necessarily assume some properties about the computer you're running it on.

Your host system must run of of these operating systems:

• macOS with a compatible Apple toolchain and the SDKs listed below. Packaged utilities target macOS 14.0; the supplied Clang executables also record a minimum of 14.0.
• Linux. Confirmed to build on Ubuntu 18.04 LTS (Bionic Beaver) and higher.

You also need some standard system utilities for compiling C code, fetching and verifying network content, manipulating text, and traversing your filesystem. To see a complete list, use make list_needed_commands. This is the full set on macOS:

$ make list_needed_commands
ar awk bash bzip2 c++ cc cd chmod cp curl find gsed gzip install ld ln make mkdir rm shasum strip tar touch xcrun xz zstd

MacOS

xcode-select --install && brew install gnu-sed zstd

Docker Desktop is needed only for Linux builds.

Unfortunately, you must install GNU sed and have it available as gsed on your $PATH to build the utils target.

Alpine

Tested on version 3.15 and higher.

apk add alpine-sdk bash curl docker tar xz zstd

Fedora

Tested on version 37 and higher.

dnf install bzip2 docker gcc make xz zstd

Ubuntu

Tested on version 18.04 and higher.

apt update && apt install build-essential curl docker.io zstd

Docker

Docker is required on both macOS and Linux for building the Linux components. This ensures reproducible builds across all host platforms.

The build uses multi-platform capabilities provided by BuildKit via docker buildx.

macOS options:

• Docker Desktop
• OrbStack (recommended - faster and lighter weight)

Note: Alternate container runtimes like Colima will not work out of the box.

NOTE As of July 5, 2023 with Docker Desktop v4.21.1 (114176), successfully building the x86_64_linux targets requires the beta feature "Use Rosetta for x86/amd64 emulation on Apple Silicon" to be toggled OFF. Until this limitation is resolved, x86_64_linux builds on Apple Silicon hosts are slow. Go make a cup of tea.

Components

Each component can be used as a make target. For example, running make dash on an x86_64 Linux computer will produce $(DESTDIR)/dash_x86_64_linux.

Common

Provided for both Linux and MacOS platforms:

• dash - A minimal POSIX-compliant shell. Sourced from gondor.apana.org.au.
• toolchain - On Linux, this is a statically-linked musl-based GCC toolchain sourced from musl.cc. On MacOS, this is the standard Apple Clang distribution.
• utils - On Linux, this bundle solely contains busybox. On macOS, it contains toybox alongside expr, gawk, grep, and tr, all from GNU.

Linux-only

• env - Sourced from GNU coreutils.
• sandbox - Sandbox environment built from source in Docker (Fedora 45). Includes glibc (dynamic linker and core libraries) and libgcc_s, plus the dash shell and env binary arranged in a directory hierarchy.

MacOS-only

• sdk - Versioned headers and metadata for macOS APIs. Not to be confused with the Tangram SDK!

macOS executable components now ship as separate <component>_aarch64_darwin.tar.zst and <component>_x86_64_darwin.tar.zst archives, matching Linux. SDK archives remain architecture-independent: macos_sdk_<version>.tar.zst, with versions 12.1, 14.5, 15.2, 26.5, and 27.0.

The host toolchain is copied from the selected Xcode/Command Line Tools installation (xcrun --find clang).

SDKs come from /Library/Developer/CommandLineTools/SDKs/.

Utilities for both architectures are built with the selected native compiler and SDK 27.0, targeting macOS 14.0.

The build inputs can be overridden with MACOS_BUILD_TOOLCHAIN, MACOS_BUILD_SDK, MACOS_DEPLOYMENT_TARGET, and MACOS_COMMAND_LINE_TOOLS_PATH. BUILD_JOBS defaults to 4 jobs per utility build. Run make clean when changing these inputs or the installed toolchain/SDKs.

Usage

The build manages the following directories:

• DESTDIR - Output artifacts ready to be included in the Tangram bootstrap module. Default: dist.
• BUILDDIR - Intermediate build artifacts. Default: build.
• SOURCEDIR - Source code, signatures, checksums. Default: sources.

The locations and contents of BUILDDIR and SOURCEDIR are not meaningful or known to the Tangram package.

Building

• all - equivalent to running make with no target defined. Build each supported entrypoint for your host platform.
• all_darwin - On macOS, package the host toolchain, build utilities for both Darwin architectures, and copy all versioned SDKs.
• all_platforms - On macOS, build both Darwin and both Linux architectures.
• <component> - Build a single component for your detected host platform.
• <component>_<platform> - Build a single component for a specific platform, if supported.
• tarballs - Create compressed tarballs for each component.
• validate_environment - Check for the existence of all required tools in $PATH. It is not necessary to call this target manually.

Cleaning

The top-level clean targets are not component-aware and obliterate entire directories:

• clean - clear DESTDIR and BUILDDIR, but retain SOURCEDIR contents.
• clean_all - clear everything. Equivalent to running the clean and clean_sources targets.
• clean_dist - just clear DESTDIR.
• clean_sources - just clear SOURCEDIR.

Additionally, each component defines cleaning targets which only remove its own artifacts, with the same semantics as above:

• clean_<component>(_<platform>)?
• clean_<component>(_<platform>)?_all
• clean_<component>(_<platform>)?_dist
• clean_<component>(_<platform>)?_sources

Omitting the platform cleans that component for all supported platforms. For example, clean_dash_dist removes every dash bundle.

Note that multiple platforms may depend on the same sources. Using a platform-specific target to clean sources will affect all platforms that share that source. For example, running make clean_dash_x86_64_linux_sources will force make dash_aarch64_darwin to re-download the source code as well.

Listing

None of these targets will catalyze any builds or downloads.

• list - Enumerate all targets that will be produced by make all.
• list_all_targets - Enumerate every single available target.
• list_needed_commands - Enumerate every utility that must be present in your $PATH to build successfully.
• list_cross_targets - On macOS, enumerate all available cross-platform distribution targets.
• list_all_platforms - On macOS, enumerate all targets that will be produced by make all_platforms. This set is the union of the targets given by list and list_cross_targets.

Docker

• docker_images - build the Docker images for Linux builds. You do not need to manually call this before building, it's provided for completeness.
• docker_glibc_images - build the Fedora-based Docker images for sandbox library builds.
• docker_stopall - stop any container this Makefile can create.
• clean_docker - Stop and remove all docker containers and images created by this Makefile.
• clean_docker_glibc - remove sandbox library Docker images.