Concept
In a component based embedded software project, the components talk to each other by reading
and writing global variables. That is a perfectly reasonable design for a target with a few
kilobytes of ram and no operating system worth the name: a write costs one store instruction,
a read costs one load, there is no message queue to size and no serialisation to get wrong.
What it costs instead is that the interface between two components exists only by convention.
Nothing in the c code says that ValueE is written by the controller and read by the user
interface; the compiler sees one extern uint16_t and is perfectly happy to let anybody
assign to it.
A convention holds for as long as the people who agreed on it remember it, and it stops holding at exactly the point where the project gets large enough for that to matter. Three failure modes recur.
An interface that exists only by convention is not reviewable. When a component is delivered by another team or another company, the only description of what it produces and what it consumes is a header file somebody wrote by hand, a spreadsheet, and eventually a conversation. There is no artefact an integrator can check before the first build, and no artefact a supplier can be held to.
A variable with no single owner is written from two places. This is the failure that
survives compilation and linking: two components both decide that they are the ones computing
SharedValue, the linker resolves both writes to the same address, and the value the
consumers observe depends on the order in which the two producers happen to run. Nothing in
the build reports it, and the symptom appears months later as an intermittent fault.
Silent drift is the slowest of the three. A component declares ValueF as an sint16
scaled by 0.1 degC per bit; six months later its author rescales it to 0.01, and the consumer
- which declared its own extern int16_t ValueF in its own header, because that is how c
works - keeps multiplying by 0.1. The software compiles, links, runs, and reports every
temperature wrong by a factor of ten. The same drift reaches the calibration engineer, because
the a2l file that describes the scaling to the measurement and calibration tool is maintained
by hand somewhere else again.
DDD removes all three by turning the interface into an artefact rather than an agreement. Every component describes the variables it produces and consumes in a small json file, DDD checks that all components agree, and then DDD - not the component authors - writes the c code that declares and defines those variables, and the a2l file that describes them to the measurement and calibration tools.
Principles
Four principles decide everything else in the tool. They are worth reading before the file formats, because most of the questions that come up later - “why can I not just declare it in my own header?”, “why does DDD refuse two writers?”, “why is the producer’s definition the one that wins?”, “why does DDD not decide what my generated header is called?” - are answered by one of them.
Every component declares its data interface explicitly
A component ships a *.ddd.json file listing every global variable it takes part in,
together with what the value is: its datatype, its physical unit, the conversion from the
raw stored value to the physical one, its limits, its initial value, whether the software has
to re-read it at every access, and, for calibration data, the axes it is defined over. Nothing
about a shared variable stays implicit, and in particular nothing about it lives in a hand
written header that only one team ever reads.
{
"scope": "output",
"definition": {
"name": "ValueF",
"kind": "measurement",
"description": "Signed measurement with a fixed point conversion",
"datatype": "sint16",
"unit": "degC",
"conversion": { "kind": "linear", "factor": 0.1, "offset": 0.0 },
"limits": { "min": -40, "max": 150 },
"init": -400,
"volatile": false
}
}
Writing the unit and the conversion down next to the datatype is not decoration: they are part
of the interface in exactly the same way the datatype is. A consumer that reads ValueF as
raw counts and multiplies by 0.01 is as broken as one that reads it as a uint8_t, and both
are equally invisible to a compiler. Because the declaration is a data file rather than c code,
DDD can compare it against the declaration made by every other component, which is the second
half of the job and the subject of Consistency checks.
The description is also the only place the information is written down. The same declaration
produces the c definition, the comment above it, the extern declaration in each consumer’s
header and the MEASUREMENT entry of the a2l, so those four cannot disagree with each other.
Changing the scaling is one edit in one file, and everything that depends on it follows on the
next build.
Exactly one component owns each variable
Each declaration carries a scope, and a variable may be declared output by exactly one
component. That component is the producer of the variable, and two consequences follow.
The first is that a second writer becomes a build error rather than a race nobody notices, and
that the error names both declarations before anything is compiled. The second is that when
two components describe the same variable differently, the disagreement has a resolution
rather than a discussion: the producer’s declaration is the authoritative one. It is the
definition that gets generated, and the diagnostic points at the consumer that deviates from
it, which is also the component that has to change. Both appear in the inconsistent
example, whose whole point is to fail:
$ ddd check examples/inconsistent/project.ddd.json
examples/inconsistent/component_b.ddd.json#component.interface[0]: error[multiple-producers]: 'SharedValue' is written by component 'ComponentB' and by component 'ComponentA'; exactly one writer is allowed
note: examples/inconsistent/component_a.ddd.json#component.interface[0]: also written here
examples/inconsistent/component_c.ddd.json#component.interface[0].definition: error[definition-mismatch]: 'SharedValue' is declared differently by component 'ComponentC' than by 'ComponentA' (datatype: uint16 != sint16, conversion: identity != linear(factor=0.5, offset=0))
note: examples/inconsistent/component_a.ddd.json#component.interface[0].definition: reference declaration
examples/inconsistent/component_c.ddd.json#component.interface[1]: error[missing-producer]: 'MissingValue' is read by component 'ComponentC' but no component declares it as output
examples/inconsistent/component_c.ddd.json#component.interface[2]: error[local-conflict]: 'Scratch' is local to component 'ComponentA' but is also declared as input by component 'ComponentC'
note: examples/inconsistent/component_a.ddd.json#component.interface[2]: declared local here
examples/inconsistent/component_a.ddd.json#component.interface[1]: warning[unused-output]: 'UnusedSignal' is written by component 'ComponentA' but read by nobody
4 errors, 1 warning
Every finding names the file, the path inside it and the check that produced it, and the findings are collected rather than raised one at a time, so a single run tells an integrator everything that does not fit. The remaining three findings above belong to the scopes and are discussed below; Consistency checks covers the full list and how a project changes the severity of any of them.
Ownership is what makes a project reviewable rather than merely consistent. For every variable there is one component, one file and one team answerable for what the value means. Comparing deliveries builds directly on that, because a variable that changed owner between two deliveries is a fact somebody needs to be told about even when both deliveries are internally consistent.
Access rules are enforced by what is generated
A rule that depends on everybody remembering it is not a rule, so DDD does not ask components to respect the declared scopes. It generates the code so that respecting them is the path of least resistance and, optionally, the only path that compiles at all.
Every component gets one header containing the declarations of that component and nothing
else, and that header is the only one a component is expected to include. There is deliberately
no project wide header offering everything: ddd_globals.h declares every object of the
project, but it exists so that the single definition file can be compiled with full prototype
checking, and it says so in its own text rather than leaving the reader to guess. Provided a
component includes its own header, a name it never declared - say BlockA, which belongs to
the user interface - has no declaration on its include path at all, so a reference to it fails
with an ordinary undeclared identifier error, at the point of the mistake, in the component
that made it.
/* Controller.h, shortened - the whole file has the same three sections */
/* outputs - written by Controller, read by other components */
/** Measurement used as the input quantity of AxisA [Hz] */
extern volatile uint16_t ValueE;
/** Signed measurement with a fixed point conversion [degC] */
extern int16_t ValueF;
/* inputs - produced elsewhere, Controller may only read them */
/** Scalar measurement with a linear conversion [%] */
extern uint8_t ValueA; /* produced by SensorHub */
/** Array measurement with four elements [V] */
extern volatile uint16_t ValueB[4]; /* produced by SensorHub */
/* locals - owned exclusively by Controller */
/** Component local measurement of the controller [%] */
extern int16_t ValueH;
Visibility alone cannot stop a component from writing to a variable it legitimately reads,
since a consumer needs the declaration in order to read it at all. --const-inputs closes
that gap by qualifying inputs const in the consumer headers, which turns a write to a
foreign variable from a convention into a constraint violation the compiler has to diagnose:
/* UserInterface.h, generated with --const-inputs */
/* inputs - produced elsewhere, UserInterface may only read them */
/** Measurement used as the input quantity of AxisA [Hz] */
extern const volatile uint16_t ValueE; /* produced by Controller */
/** Signed measurement with a fixed point conversion [degC] */
extern const int16_t ValueF; /* produced by Controller */
Note
The definition in ddd_globals.c stays non-const, because the producer has to be able to
write it. Declaring the same object const in one translation unit and non-const in
another is a constraint violation in strict c, even though the usual embedded toolchains
accept it and generate exactly the code one expects. That is why the stronger enforcement
is opt-in rather than the default: a project decides for itself whether its toolchain and
its coding standard allow it.
The same principle runs through the rest of the tool. A component local variable is defined in
ddd_globals.c like any other, so that the project still has exactly one definition per
object, but its declaration appears in the header of its owner and nowhere else. An axis that
a curve refers to is written to the a2l even when the axis itself was marked as not exported,
because an AXIS_PTS_REF without the AXIS_PTS it points at would not be a valid a2l file
and the calibration tool would reject the whole thing. In both cases the rule is carried by
what comes out of the generator, not by a sentence in a document that somebody has to have
read.
DDD owns the data, the project owns the presentation
What the generated c says follows from the declarations. What it looks like does not: the
comment style, the banner, the include guards, the section headings and the names of the files
themselves are house style, they differ between projects, and nothing in a description file
decides them. A generator that imposed its own would either be argued with or be worked around
with a post-processing script, so DDD does not: the c sources are rendered from jinja2
templates the project provides. ddd generate requires --template-dir and falls back to
nothing, ddd templates-dir prints a working set of examples to copy into a project and
change, and a project renames a generated file by renaming the template that produces it. Even
the file names above are the example templates’: ddd_globals.h is called that because a
template is called ddd_globals.h.jinja2. Templates describes the rules in full.
Owning the presentation is not owning the rules. A component template is handed the
declarations of its own component - already grouped by scope, already spelled in c, already
qualified const where --const-inputs says so - and the ownership, the visibility and
the datatypes are settled before any template runs. The example templates write plain c
comments because a comment convention is exactly the kind of decision this principle leaves to
the project; one that documents its generated code differently, or not at all, changes the
comment in the template and nothing else.
The a2l file is the exception that proves the point. Its structure is dictated by ASAM rather
than by a house style: a measurement and calibration tool rejects a COMPU_METHOD in the
wrong place no matter whose coding standard produced it. There is nothing for a project to
decide there, so there is no template option for the a2l either - that backend carries its own
templates and stays internal.
Vocabulary
The terms below are used with these meanings throughout the documentation, the diagnostics and the json schemas. They describe the roles the data plays, not the kind of device the software runs on: DDD is not tied to any industry, and “measurement” or “calibration parameter” say what a value is for, not what it controls.
term |
meaning |
|---|---|
project |
a named set of components and/or sub-projects, described by a file whose top level key
is |
component |
a software unit with an explicitly declared data interface, described by a file whose
top level key is |
declaration |
one entry of a component interface: a scope, an optional c preprocessor condition and a definition. A component takes part in a variable by declaring it, and only by declaring it. |
data object |
the thing being declared: a measurement, a parameter, a value block, a curve, a map or an axis. “Variable” and “data object” are used interchangeably; the second is the more precise word once calibration data is involved. |
scope |
ownership and visibility of a data object with respect to the declaring component, one
of |
conversion |
the rule that maps the raw value stored in the target to the physical value a human reads: the identity, a linear factor and offset, or an enumeration. |
producer |
the component that owns a data object, that is the one which declared it |
data dictionary |
the resolved result: every object with its owner, its consumers, its shape and its limits worked out. It is the contract between the checking front end and the output backends, and DDD publishes it - see Data dictionary. |
scope
A declaration states what the declaring component does with the object, and that single word is all DDD needs in order to work out ownership, visibility and the direction of every interface in the project.
scope |
meaning |
|---|---|
|
the component reads the object, and some other component has to produce it. An input
that nobody produces is reported as |
|
the component owns the object, and exactly one component may do so. Its declaration is the authoritative one, and the object appears in the headers of the consumers annotated with the name of its producer. |
|
the component owns the object exclusively and no other component may use it. A second
component declaring the same name is reported as |
For a measurement, output has the obvious reading: the component writes the variable at
run time and everybody else reads it. For calibration data - parameters, value blocks, curves,
maps and axes - the software never writes anything at all, since those objects are generated
const. output there means that the component provides the data and that other
components may read it; the writing is done by the calibration tool, from outside the
software.
Whether the software has to notice that writing is the other half of the answer, and it is
what every definition states as volatile. An object a calibration tool tunes while the
software runs is generated const volatile, because const alone lets the compiler use
the initial value in place of a read wherever it can see it - within one translation unit at
every optimisation level, -O0 included, and across them under -flto - and, where the
load does survive, still lets it serve two reads from one of them. Either way the tool writes
a value the software does not pick up. The qualifier
costs the read only section: gcc moves a const volatile object out of .rodata into
.data, which on a flash target is a question the linker script has to answer. An object
that is never tuned online states false, is generated plain const and stays in read
only memory. DDD states no preference between the two and reports nothing about the choice;
it renders what the description says.
The demo project uses both readings. AxisA is declared output by the controller and
input by the user interface, which defines its own curve CurveB over the controller’s
break points - the axis is stored once and shared, which is what the a2l calls COM_AXIS.
The controller’s ParameterA, on the other hand, is local, because it parametrises the
controller and nothing else:
$ ddd list examples/demo/demo.ddd.json
VARIABLE KIND DATATYPE UNIT SHAPE INIT PRODUCER CONSUMERS
AxisA axis uint16 Hz [6] [...] Controller UserInterface
AxisB axis uint8 % [4] [...] Controller (local) -
BlockA value_block uint8 - [8] [...] UserInterface (local) -
CurveA curve uint16 ms [6] [...] Controller (local) -
CurveB curve uint8 % [6] 200 (= 100 %) UserInterface (local) -
FlagA measurement boolean - - 0 SensorHub EventLogger
MapA map sint8 % [4][6] [...] Controller (local) -
ParameterA parameter uint16 Hz - 3200 (= 800 Hz) Controller (local) -
StateA measurement uint8 - - 0 (= STATE_OFF) Controller UserInterface
ValueA measurement uint8 % - 0 (= 0 %) SensorHub Controller
ValueB measurement uint16 V [4] 0 (= 0 V) SensorHub Controller, UserInterface
ValueC measurement float32 degC - - SensorHub UserInterface
ValueD measurement uint16 - [8] - SensorHub (local) -
ValueE measurement uint16 Hz - 0 (= 0 Hz) Controller UserInterface, EventLogger
ValueF measurement sint16 degC - -400 (= -40 degC) Controller UserInterface
ValueG measurement uint16 V - 1000 (= 1 V) Controller UserInterface
ValueH measurement sint16 % - 0 (= 0 %) Controller (local) -
ValueI measurement uint32 - - 0 UserInterface EventLogger
ValueJ measurement uint8 - - 0 EventLogger UserInterface
ValueK measurement sint8 - [3][4] [...] EventLogger (local) -
local is the normal choice for calibration data that only tunes its owning component, and
it is worth preferring over output whenever it applies: it keeps the object out of every
other component’s header, and it lets the tool report a new user as a mistake instead of
silently accepting it.
Note
Scope is a property of a declaration, not of an object. The same object is output in
the file of its producer and input in the file of every consumer, and each side states
the full definition. That redundancy is deliberate: it is precisely what allows DDD to
notice that the two sides have drifted apart, which a single shared declaration could never
do.
Position in the build process
DDD is a command line tool, so that it can be driven from make, from CMake, from a batch file or from a ci job, and it runs twice per build. Understanding why there are two runs explains most of what the build system integration does.
The first run happens before anything is compiled. It reads the project description, follows the includes, resolves the declarations of all components into one data dictionary, runs the consistency checks and - only if they pass - renders the project’s c templates and writes a first a2l file. From then on the compiler has both the definitions of every global variable and one header per component:
$ ddd generate examples/demo/demo.ddd.json -o build/gen -t examples/templates
wrote build/gen/ddd_globals.c (created)
wrote build/gen/ddd_globals.h (created)
wrote build/gen/ddd_types.h (created)
wrote build/gen/Controller.h (created)
wrote build/gen/SensorHub.h (created)
wrote build/gen/UserInterface.h (created)
wrote build/gen/EventLogger.h (created)
wrote build/gen/DemoDevice.a2l (created)
The second run happens after linking, and it exists because of one piece of information that does not exist before then: the address of every object in the target. A measurement and calibration tool reaches a variable over the debug or xcp interface by address, so an a2l without addresses describes the software correctly but cannot be used to measure it. The addresses are decided by the linker, so DDD is handed a symbol to address map extracted from the linker output and rewrites the a2l with the real values. The map is a flat json object, with the addresses written in decimal or hexadecimal:
{ "ValueE": "0x20000100", "AxisA": "0x08004000", "CurveA": 134234112 }
$ ddd generate examples/demo/demo.ddd.json -o build/gen -t examples/templates --address-map build/addresses.json
unchanged build/gen/ddd_globals.c
unchanged build/gen/ddd_globals.h
unchanged build/gen/ddd_types.h
unchanged build/gen/Controller.h
unchanged build/gen/SensorHub.h
unchanged build/gen/UserInterface.h
unchanged build/gen/EventLogger.h
wrote build/gen/DemoDevice.a2l (updated)
The second run regenerates everything but writes only what actually changed - which, since the declarations did not move, is the a2l alone. That property is not a convenience for the console output. A generated file whose content is identical is left untouched on disk, so its time stamp does not move and the build system does not rebuild the whole project after the address import. For the same reason the generated files carry no time stamp of their own, and regenerating from unchanged inputs produces a byte identical result, which is what makes a delivery reproducible.
Before the address map is applied every address in the a2l is 0x00000000; afterwards the
objects named in the map carry their real address, and the ones that are not named keep zero.
SYMBOL_LINK is emitted in both cases, so a project that would rather patch the addresses
into the a2l with a separate tool after linking can skip the second run entirely and still
have the symbol names in the file:
/begin MEASUREMENT ValueE "Measurement used as the input quantity of AxisA"
UWORD CM_LIN_HZ 0 0 0 8000
ECU_ADDRESS 0x20000100
SYMBOL_LINK "ValueE" 0
/end MEASUREMENT
Neither run needs a compiler, a target or a calibration tool to be present, which is what makes the tool usable from a pipeline that only checks: Command line interface describes the exit codes and the machine readable output a ci job consumes, and Build integration reduces the whole sequence above to two calls in a CMake project.
Note
A project that has no use for an a2l at all can pass --no-a2l and use DDD purely as the
owner of its global variables; a project that only wants the verdict, for instance in a
merge request pipeline, runs ddd check and generates nothing.
How the tool is put together
The internal structure of DDD deserves a paragraph in a conceptual page because it is visible from the outside. It is the reason the tool can be relied on to say the same thing about a project in c as it says in a2l, and the reason a project can generate an output format DDD does not ship.
Everything DDD knows about a project passes through a single artefact, the data dictionary.
The front end - the file formats, the loader that follows the includes and the analysis that
works out ownership, agreement and references - produces it, and mentions neither c nor a2l
anywhere. The backends consume it, and never touch a description file, a glob or a check. The
c backend knows what a uint16_t is and has never heard of a COMPU_METHOD; the a2l
backend knows what a UWORD is and has never heard of an include guard.
The two differ in where their templates come from, and that difference is the fourth principle in code. The c backend is handed a template directory and renders whatever it finds there, so the shape of the generated c is outside the tool; the a2l backend is self-contained, because the shape of an a2l file is outside the project.
Two things follow from that split. The dictionary is published: ddd dump writes it out
as json and ddd schema dictionary prints its schema, so a generator DDD does not ship - a
header for another language, a csv for a test bench, an ARXML - can consume the resolved data
of a project without depending on anything inside the tool. And a backend is nothing more than
an object with a name and a generate(dictionary, output_dir) method
(the Backend protocol of ddd.backends.base), so adding an output format means adding a
package next
to the two existing ones and listing it where the generate command assembles its backends,
and changing nothing else.
The boundary is not a matter of intent. The test suite walks the import graph and fails the
build if the front end imports a backend, if a backend reaches into the loader or the analysis,
or if one backend imports the other; it also fails if a spelling that belongs to a single
output format - uint16_t, UWORD, AXIS_PTS - appears anywhere in the file formats.
The layering therefore cannot rot silently, and Developer documentation describes those
guards in detail.
One last property of the front end is easier to appreciate here than in a reference page: the loader and the analysis never raise on a bad project. They collect as many findings as they can in one run and report them together, because an integrator assembling a dozen components wants the list of everything that does not fit, not the first thing that did not.