CEP-2: A C API / FFI for Native Cob Extensions

1. Abstract

This proposal adds a native C API (cob.h) and a new extern statement so a .cob script can call functions compiled from C (or anything else that can export a C-compatible symbol), and so farmer packages can eventually ship that C code as a native extension. cob_interp loads the extension dynamically at runtime; popcorn_comp, which already transpiles to C and spawns a real C compiler, links against it directly at compile time instead.

2. Motivation

Cob's standard interpreter can currently only do what's built into cob_interp.c itself — SQLite, the native window, and Tcl/Tk (see the v0.0.5 release) all had to be added as new C code and new keywords directly inside the interpreter's own source file, by someone with access to that repository. There is no way for a third party to add a new capability to Cob without a PR against the core interpreter.

A C FFI fixes that: it lets anyone with a C compiler wrap an existing C library, a performance-critical routine, or a hardware/OS-specific facility, and make it callable from a .cob script without touching cob_interp.c at all. Combined with farmer, this is what actually lets a package ecosystem grow past what the two or three people with write access to the core interpreter have time to build in.

3. Specification

3.1 The value model this has to bridge

Cob has exactly two value kinds today (see v0.0.4's string support): an integer (long) and a string (char*, owned/refcounted by the interpreter). There are no arrays, structs, or function values. The FFI's job in this proposal is only to bridge those two kinds across the C boundary — not to give Cob arrays or structs, which is a separate, much bigger change and explicitly out of scope here (see 3.5).

3.2 cob.h: one fixed calling convention, not one C signature per function

Every native function callable from Cob has to expose exactly the same C signature, regardless of how many arguments it takes or what it does. This is the part of the design that makes the rest of it tractable (see Rationale for why):

/* cob.h -- the only header a native Cob extension needs */

typedef enum { COB_INT, COB_STR } CobValueKind;

typedef struct {
    CobValueKind kind;
    union {
        long        i;   /* COB_INT */
        const char *s;   /* COB_STR -- NUL-terminated, owned by the caller;
                             copy it if you need to keep it past this call */
    } as;
} CobValue;

/* Every extern function has this exact signature. argv[0..argc-1] are the
 * arguments as written at the Cob call site, left to right. */
typedef CobValue (*CobFFIFunc)(CobValue *argv, int argc);

/* Convenience constructors for a native function's return value */
static inline CobValue cob_int(long v)        { CobValue r; r.kind = COB_INT; r.as.i = v; return r; }
static inline CobValue cob_str(const char *v)  { CobValue r; r.kind = COB_STR; r.as.s = v; return r; }

A native extension author writes ordinary exported C functions matching CobFFIFunc's signature, e.g.:

/* math_ext.c -- compiled to math_ext.so / math_ext.dll */
#include "cob.h"

CobValue calculate_fast(CobValue *argv, int argc) {
    long x = argv[0].as.i;   /* real code should check argc/kind first */
    return cob_int(x * x + 1);
}

3.3 The extern statement

A new statement, one per native function, naming the library it lives in and the C symbol to bind to it:

extern calculate_fast(x) from "math_ext.so"

set result = calculate_fast(42)
pop(result)

This is close to the example in the original request, with one deliberate change: there's no separate return-type annotation (the request's logic) written in Cob syntax, because CobValue is already a tagged union — the native function decides at runtime whether it's handing back an int or a string, the same way sql_query() already does today. Parameter names (x above) are written for readability but carry no type information either, for the same reason: there's nothing to declare, since every argument is already just whichever CobValue it evaluates to.

extern resolves the library and symbol immediately, at the point the statement runs, not lazily on first call:

3.4 Cache format impact

extern is a new statement kind, so it needs a new entry in StmtKind, a writer/reader pair in the .strawberry cache, and (per the shared-constant fix that came out of the v0.0.5 cycle) a bump of the single STRAWBERRY_MAGIC now centralized in common.h. This proposal explicitly calls that out up front rather than leaving it implicit, since a stale copy of that exact constant in popcorn_comp.c has already caused two real, silent CI failures this project has had to debug — see the v0.0.5 release notes' "Fixed" section. Any implementation of this CEP should ship the cache-format bump and the round-trip test described in 5 in the same commit, not as a follow-up.

3.5 Non-goals for this proposal

4. Rationale

Why one fixed CobValue argv[] signature instead of a real native calling convention?

The alternative — generating a distinct C trampoline per distinct argument-count/type signature, or pulling in something like libffi to construct calls at runtime — is a lot more implementation surface for a language that only has two value kinds to begin with. Using one fixed signature for every extern function is the same choice Lua's lua_CFunction, Python's C API (PyObject* args), and Node's N-API all made, for the same reason: it turns "generate correct calling code for arbitrary signatures" into "generate one call to one function pointer," every time, regardless of arity. The cost is that a native extension author writes a few lines of argument-unpacking boilerplate per function instead of getting native argument types directly — a reasonable trade for how much simpler it keeps both cob_interp and popcorn_comp's side of this.

Why does extern resolve eagerly, not on first call?

A missing library or misspelled symbol name should fail at the exact line that named it, not at some later, possibly-conditional call site that might not even execute on a given run. This matches the existing "fail clearly and immediately, don't defer to a confusing later symptom" pattern already in this codebase — --no-gc is refused outright rather than harvest() silently no-oping, for the same reason.

Why does popcorn_comp link statically/dynamically instead of also using dlopen at runtime?

popcorn_comp already spawns a real C compiler (Zig's zig cc by default) to produce a genuinely standalone native executable — that's the entire point of having a native compiler rather than only an interpreter. Making a compiled binary still reach for dlopen at runtime would reintroduce a runtime dependency (the extension's shared library has to be found on disk again, on the end user's machine) that a real link step avoids entirely. cob_interp doesn't have this option — it has no compile step of its own — so it's the one side that has to use dlopen/LoadLibrary.

5. Backward Compatibility

• extern is a brand-new keyword with no collision against any existing Cob keyword or identifier rule — a script that doesn't use it is completely unaffected, in every existing build (cob_interp, cob_interp_window, cob_interp_full, cob_interp_db).
• The .strawberry cache format needs a version bump (3.4) for the new statement kind. Per the single-source-of-truth fix already in common.h, that's one constant to change, not two — and this proposal explicitly requires the writer/reader pair and the cache-format bump ship together, with a round-trip test (open a script using extern, run it twice, diff the output) added to CI in the same change, exactly mirroring the test already added for the raygui widget keywords in v0.0.5.
• cob_interp gains a new link dependency on Linux: -ldl for dlopen()/dlsym(). Windows needs nothing extra (LoadLibraryA/GetProcAddress are in kernel32, already linked by default). No new autoconf/configure step is introduced anywhere — consistent with the project's existing "zero new configure steps" stance from the v0.0.5 cycle.
• popcorn_comp gains new codegen for extern; scripts that don't use it compile exactly as before. A script that does use extern could not previously be compiled by popcorn_comp at all (it wasn't valid syntax), so there's no existing compiled-binary behavior to preserve here — this is purely additive capability, not a change to any existing one.