Why?

::info about this page: The “Why?” document outlines the reasons why this project is needed, why it’s important to do it this way, and the essential goals and vision that define our priorities and how we make our choices.

It’s a useful document if you’re considering whether to get involved, or wondering what kind of efforts are well-aligned. :::

:::notice This page is meant to replace and deprecate the “Vision” page, but both are still shown until content migration and rearrangement is complete. :::

“Why?” can be addressed at a variety of levels:

The Mission Statement – the specific objectives
The Vision – why we think these things matter
The Strategy – the major directions and key technological choices we take to execute this mission and make the vision happen.

Mission

The Timeless Stack is a platform for predictable computing.

We are building tools to match the following criteria:

declarative, enumerable dependencies (documents you can read without executing) and clear manageable supply chains
hermetic build environments, and other exacting and executable criteria to verify the above,
and doing this with a system designed from the ground up with content-addressable elements, because this makes many problems solvable without introducing coordination, meaning it naturally works well in decentralized environments.

We’re doing this because these three design criteria, together, should result in an ecosystem of tooling that’s more composable, more comprehensible, more collaborative, and more experimental-friendly (think: bazaar over cathedral) than any other possible set of tooling which elides any of these three criteria.

Vision

“Vision” describes the highest-level, most abstract goals – concerning where we want to be, without regard for how we get there (that’s what the Mission Statement is for).

motivation

Technology needs to be comprehensible. Computers need to be predictable.

Our whole civilization advances faster when our tools are comprehensible and maintainable. The integrity of our society relies upon being able to trust the systems we rely on for our communication, coordination, and increasingly even the control of our physical environment.

Or to flip it around:

Technology should not be enigmatic. Computers should not house demons.

If they are, we lose control of not just them, but everything we build that depends on them: eventually culminating in losing control over our societies, our science, and ultimately, ourselves. Comprehensible technology is not a nice-to-have: it’s an evolutionary imperative.

path

We need reproducible builds to make that happen.

We need hermetic workspace and supply chain management to make that possible, detectable, auditable, and maintainable.

And we need those things to work well in decentralized environment: critically, coordination (e.g. in naming and tagging things) needs to be both possible and well-described. At the same time, it’s ideal if coordination systems are articulated as their own independent processes atop a reproducible environment themselves. This kind of coordination-optional layered design is important because it enables development of workflows and experimentation on coordination strategies in a decentralized, non-prescriptive way.

We need, in short… the things the Timeless Stack mission aims to produce.

Seealso: documentation on hermetic computation, immutable-everything and reproducible resolve.

success criteria

We need tools that can build anything, and contain any toolchain. Our solutions should be universal. “Universal” here means not language-specific, and not siloed into a “distro” which requires bidirectional coordination to join.

Our tools should be highly reliable. Every part of our tools should be clearly able to operate usefully and predictably without interactive access to the global network.

Our tools should reinforce, encourage, and highlight ways for users to pursue their own developments in ways which are aligned with our mission and vision. Specifically: when some process being handled in the Timeless Stack is non-reproducible, our tooling should detect it, and make it clearly evident to the authoring user, who should then have the information which empowers (and encourages) them to fix it.

Our tools should work the same no matter who is using them, when, or where. Our tools should minimize presumptions; our tools should identify and maintain environments rather than expecting them.

There should be a self-reinforcing energy in the user experience of these tools: we want tools that work better when building reproducible things, and make it clearly evident the authoring user when reproducibility has been lost. The net effect of the tools should be to make it immediately clear, within short feedback loops, that reproducibility provides value; it should make the feedback loops more obvious; and it should draw attention to those feedback loops in such a way that users are encouraged to improve their own systems over time.

Strategy

familiar tools

It’s important that our tools be reasonably familiar and applicable to presently popular computing environments.

Specifically, the Timeless Stack narrows is primary target and scope to something which can summarized as “must support linux-like systems” and “must pass state primarily by familiar ‘file’-like abstractions”. (In other words – “it should taste like containers” – this is a proven path to successful adoption that lets others use what they know and pivot towards our tooling incrementally. Incremental is good.)

This scope may extend to more systems in the future, but will start here.

anything not everything

The Timeless Stack tools need to support anything – anything you want to put in them, with no special consideration required – rather than supporting “everything”, but needing plugins to do it.

(This stands in contrast to most other build tools, which try to embrace the world by neverending systems of plugins.)

breaking down the problem

It’s important to break problems down correctly before trying to solve them.

To achieve the mission of the Timeless Stack, five major sets of problems that need to be addressed:

Hermetic execution environments must be provided.
The data and software provisioning for the hermetic environment must be declaratively describable, and reproducibily provisionable. (In practice: the input data must be content addressable, so that is immutability and reproducibility is efficiently auditable.)
Coordination systems must be available describing connected graphs of executions and outputs. These are necessary both for authoring graphs of intent, and as a precipitatable document which graphs actual outcomes (and can be repeated).
Coordination must be extensible to a community: these systems need to work both in the context of collaborators with direct trusted communications (e.g. projects and close-knit teams – which is relatively easy to handle) and in the context of collaboration at large (e.g. uncoordinated swarms of projects at scale on github, etc – this is much harder!)… and this coordination must continue to be scalable without compromising on the Timeless Stack mission statements regarding hermeticity and reproducibility.
It’s useful to have a shared standard for “Responsible Packaging” – some community standards for how to package and ship software such that it works well within immutable systems and works with minimal environment presumptions and transitive dependencies.
It’s useful to provide all these systems in applications that are approachable to end-user developers and provided immediate value… for example, CI, new “app stores”, etc, all of these are useful. Ideally, given the platform core, building these should become near-trivial!

Some of these problems easier to address than others, e.g. Part 1 can be done nearly off-the-shelf with existing components. Part 2 needs a significant amount of glue code at present. (The systems exist, but one must write code to connect them.) Part 3 is a significant design challenge. (It’s difficult to pull off coordination systems without shrugging into some system that whimically refers to “latest”. The siren song of “latest” is… strong.) Part 4 is pretty much a leading edge research problem in distributed workflow UX. We hope to address this in the Timeless Stack, because it needs to be done. (At the very least: we aim to provide a fertile ground with APIs in which others can experiment with coordination systems.) Part 5 is a quest to introduce community standards. (A convention here will also ideally be well-supported by tooling, such that it becomes a defacto standard because it’s a path of least resistance.) Part 6 is actually quite open-ended, and we hope to see lots of independent projects flourish there!

All of these problems are either addressed by, or are adjacent to, the Timeless Stack mission. Now that they are identified broken down, they should become more tractable.

composable tools

In each of the layers of problems described above, ideally, the problem should be solved by a layer of tooling targetted at that problem.

These tools should work well together, but also be usable independently.

See the Tools Reference pages for an index of each of the tools we are currently develop.

API and checkpoint driven

Each of our tools should work declaratively, be easily driven by APIs (not just CLIs and sloppy glue), and be “checkpoint driven” – which means the data passed between the APIs should be possible to serialize, and – at that instant – inspect it, save it, repeat it, shift it to another machine, whatever. Code should not be passed between processes; documents should.

“API” should mean simple common formats – JSON piped to stdin and JSON streamed to stdout is a solid API. Our tools emit human-targetted messaging by default, but adhere to this API structure when instructed to via --format=json.

See the design/layers page for more about our API layers.

hash all the things

One key theme we’ll find recur overall in the implementation tactics of the Timeless Stack so often that it might as well be called strategy: we’re going to hash everything.

Hashes let us build content-addressable systems, which work well in decentralized systems;
hashes give us ways to dodge naming problems which would otherwise introduce coordination issues, which makes interoperability easier even for people working independently;
and hashes create quick ways to detect if data is identical at massive scales, which is critical to reproducibility detection.

Since all three of these properties are so critically useful, executing strategic design with a “hashes first” philosophy gives the Timeless Stack a distinct style – and a distinct advantage.