Case study
Aurelia teaches AWS, LSAT and quant by conversation. It speaks, it listens, it draws on a whiteboard, and it remembers what you got wrong last week. It was compiled, signed and installed on Fedora, with no Mac anywhere in the process — and the harder problem turned out to be debugging an app you are completely unable to see.
The tutor's replies arrive as a token stream with hidden tags woven through the speech. The app takes each one as it arrives rather than waiting for the reply to finish, so the board fills in while it is still talking.
# A reply, as it streams in. Speech is spoken; the tags are not. # Right, so S3 cross-region replication is not a backup strategy, # it is a disaster recovery one — # <board>RPO=15min RTO=4h</board> # <log>S3 replication strategy|correct</log> # <note>Confused retention with RPO — revisit D3.</note> # <todo>Redraw the async replication diagram tomorrow</todo>
There are 238 days of AWS and LSAT material and 24 weeks of quant checklist behind it. The answer key is withheld from the prompt on purpose, so the tutor has to hear what the learner actually says rather than reading the right answer off the page.
It knows the material, what you miss, your streak, which syllabus sections you have covered, and what you said last time. A miss sends the topic back tomorrow; a hit moves it up a ladder of 1, 3, 7, 16 and 35 days.
262
days and weeks of material parsed and tested
74
tests, running natively on Linux
73
distinct points the app can log
61%
of 5,125 lines of Swift have no UI in them, so all 74 tests run on Linux
Here is the part that made this different from everything else I have built on Linux.
xtool will compile, sign and install an iOS app on a Mac-less machine, and then
the two things you normally rely on stop working.
There are no device screenshots. Xcode 27 ships no Developer Disk Image, so nothing
can capture the screen. xtool launch cannot start apps on this device either, so
the app has to be launched by tapping its icon like a person would.
And NSLog goes nowhere. There is no console to watch.
So the first useful thing I wrote was not the tutor. It was a log file:
# Every decision, every format, every chunk, appended to Documents/ DiagLog.event("audio.activated", ["try": 1]) DiagLog.event("audio.format", ["channels": 1, "sampleRate": 48000]) DiagLog.event("turn.begin", ["chars": 41]) DiagLog.event("http.first", ["ms": 214]) DiagLog.event("stt.partial", ["final": true, "text": "cross region replication"])
Then it gets pulled off the device over USB, because there is no other way in:
mkdir -p /tmp/aurelia
ifuse -u 00008130-0006516C02C0001C --documents ch.stanley.aurelia /tmp/aurelia
cat /tmp/aurelia/aurelia.log
Not decoration. The turn that hung silently for fifteen minutes was one log line away from being found, and without this file it would still be hanging.
Three bugs that were completely invisible from the code, and that each look like nothing at all when you read the source.
The app talks and it listens, and AVAudioSession allows exactly one thing at a
time. When the voice-playback queue claimed the session and set its category on initialisation,
the microphone could no longer open — and the failure was a single line:
Session activation failed
Not a crash, not a hang, not an empty state. The recogniser now walks a ladder of category and option combinations and takes the first that actually activates, because which one is accepted depends on what the system is already doing with audio.
installTap raises, and it does not throwThe canonical Swift snippet is try AVAudioSession...installTap(...), which makes
it look like a failure a catch can handle. It is not one. Reading
inputNode.outputFormat(forBus:) before the session is configured and activated
reports 0 Hz and 0 channels, and installTap raises on that rather than returning
an error. A catch cannot save you. Check the values instead.
This one was my mistake, and it is the reason I am writing the section. I had listed the microphone as unverified and queued up an audio debugging session. The log had been saying the whole time:
perm.speechStatus raw=3 perm.micStatus granted=true audio.activated try=1 audio.format channels=1 sampleRate=48000 audio.engineRunning stt.partial final=true text="cross region replication"
Recognition had worked. The transcript was reaching the app and the composer was dropping it. Chasing the audio stack would have found nothing, and the bug I was about to fix would have still been there.
Everything testable lives in a target with no SwiftUI and no UIKit, which means
swift test runs natively on Linux in about sixteen seconds. Nothing is mocked,
because there is nothing to mock: the tag parser, the curriculum loader, the HTML reader, the
prompt assembly, the progress ladder and the SSE reader are all Foundation.
Sources/StudyBuddyCore/ Foundation only — this is what the 74 tests run
TagParser.swift the speech / side-effect split
ContentStore.swift reads the bundled curricula
MiniHTML.swift just enough HTML for fixed generated pages
PromptBuilder.swift the persona and prompt assembly
ProgressStore.swift SQLite, a direct port of the Python schema
ProgressMath.swift the ladder, extracted so it can be tested
SSEReader.swift one AVAudioSession-safe reader
Sources/StudyBuddy/ the app. Builds only through xtool.
BoardView.swift the whiteboard; one web view, for mermaid only
SpeechRecognizer.swift on-device listening
The whiteboard is the part I like most. The model can draw a table, a table of values, a
code block, a formula or a mermaid diagram, and all five render natively — except mermaid,
which gets the single WKWebView in the app, scoped to that one case and nothing
else. An unrecognised board kind falls back to a plain list, because the model does invent
kinds, and rendering nothing is the worst possible answer.
export PATH="$HOME/.local/swift/usr/bin:$PATH" # Swift 6.4.1 is not on PATH by default bash tools/stage-content.sh # copy the curricula in bash tools/inject-key.sh # bake the provider key in swift test # 74 tests, on Linux xtool dev build # → xtool/Aurelia.app xtool dev build --configuration release --ipa # → 11.8 MB xtool install xtool/Aurelia.ipa
| Requirement | Value |
|---|---|
| Swift | 6.4.1, Linux, no Mac toolchain |
| xtool | 1.22.2 |
| SDK | built from Xcode 27 with xtool sdk install --slim |
| Signing | a free personal team, on Linux |
| Mac required | none |
The provider key is compiled into the binary by a script that writes a gitignored file. That is a deliberate trade for a personal app and it is the wrong default for anything else — anyone with the .ipa has the key. For a release build this belongs in the Keychain.
Listed because a case study that only lists achievements is advertising, not engineering.
403 can only be used from within OpenCode. That is a
restriction on the client, not on the key. A workaround exists — shell out to the opencode
CLI, which is inside OpenCode and therefore allowed — but it needs a machine switched on,
so it is not the default.openGym — Duarte Santos. The iOS build established that a SwiftUI app compiles on Linux with xtool; the same techniques carry over here, and that case study covers the build itself in more depth.
xtool — the cross-platform Xcode replacement that makes any of this possible.
Chatterbox — Resemble AI,
MIT. The only free speech model with real paralinguistic control: it understands
[laugh], [chuckle] and [sigh] as text, and has an
exaggeration slider. A tutor's prosody carries meaning the transcript does not, so the engine
is handed a mood rather than a string — a hint slows down, praise lifts slightly, and a
correction stays level rather than dropping, which reads as disappointment.
study-mentor — the Python tutor Aurelia is a rewrite of. The tag protocol, the prompt persona, the SQLite schema and the spaced-repetition ladder are all ports of it.
Aurelia is MIT, © Stanley Sujith Nelavala. The AWS and LSAT curricula are MIT. The quant checklist is Apache-2.0. Chatterbox is MIT. openGym is AGPL-3.0, and is linked here rather than vendored.