JackFrostbyte/rgb-aura
1
2
Fork 0
Code Issues 24 Pull requests Projects Releases 2 Packages Wiki Activity Actions

RFC: Project direction for v1.2+ — LLM-assisted Aura making studio #43

New issue
Closed
opened 2026-05-11 00:51:24 -04:00 by McJuniorstein · 5 comments
McJuniorstein commented 2026-05-11 00:51:24 -04:00
Collaborator
Copy link

Context

v1.0 is shipped. v1.1.X scope is foundation work (bug fixes, test scaffolding, dead-file cleanup) and does not depend on this decision. This RFC settles what rgb-aura aims to be in v1.2+.

Natural follow-on to docs/llm-project-workflow.md: that doc covered using an LLM to build rgb-aura. This RFC asks whether the next chapter should be a tool where the LLM helps users design their own RGB animations — same human-in-the-loop philosophy, one layer up.

Two avenues considered

Avenue 1 — Full GUI Aura making studio. Build a Linux equivalent of ASUS Aura Creator: timeline editor, preview canvas, profile manager, color pickers, hardware test loops. Familiar paradigm, large scope, year-plus of work, lands as "almost as good as OpenRGB GUI + Aura Creator clone."

Avenue 2 — LLM-assisted Aura making studio. rgb-aura becomes the substrate for LLM-driven design. User describes the animation they want; an LLM produces working animation code using rgb-aura's API + a shipped context pack (primitives, examples, OpenRGB conventions, canonical prompt). Scope is bounded — the "studio" is the LLM. Plays to rgb-aura's existing strengths (safety architecture, modular animation registry, dry-run mode are exactly what an LLM needs to be productive). Multiplatform-friendly via OpenRGB + Python.

Prior art (May 2026)

Independently researched across three LLMs before writing this RFC. Full receipts in docs/research/2026-05-10-prior-art-{perplexity,gemini,claude}.md. Summary:

  • Skydimo (skydimo.com) — proprietary, cross-vendor (1000+ brands), explicitly advertises "describe your idea in natural language, AI instantly generates the corresponding lighting effects." Closest functional match. Closed-source, not OpenRGB-based, AI embedded in their controller (not portable).
  • Lumia Stream (lumiastream.com) — commercial, ChatGPT-plugin-based natural-language RGB animation. Streaming-focused market. Same UX pattern, different segment.
  • Corsair iCUE Murals — commercial AI lighting, but image/video-driven, not natural-language-driven. Vendor-locked to Corsair.
  • Razer Adaptive Immersive Experience (GDC 2026) — runtime-adaptive lighting/haptics/audio. Not a design tool.
  • Arduino Cloud AI Assistant (Anthropic + Arduino, mid-2025) — "describe it and Claude builds the sketch" for microcontrollers. Closest pattern match for LLM + hardware + natural-language code generation. Vendor-integrated SaaS, single-LLM, different hardware target.
  • NVIDIA Project G-Assist — SLM controlling PC APIs including RGB as one of many actions. General PC assistant, not a design tool.
  • TD-OpenRouter (TouchDesigner, May 2026) — closest architectural analog of the context-pack pattern in a different creative domain.
  • p5.js v2.0 community context packs (HuggingFace, 2025+) — informal but established pattern of shipping API-doc + prompt bundles for LLM-assisted creative coding.

Honest novelty verdict: partially novel. The high-level UX is no longer unprecedented — Skydimo and Lumia Stream have shipped commercial versions. Genuinely novel in implementation:

  1. Open-source (vs. all named commercial competitors)
  2. Built on OpenRGB (vs. proprietary protocol stacks)
  3. Portable context pack users hand to any LLM, including local models (vs. AI embedded in vendor product — you can't take Skydimo's AI to Claude or a local Llama)
  4. Linux-first (vs. Windows-first commercial offerings)
  5. Natural-language-driven (vs. iCUE Murals image-driven)

That's a defensible niche.

Recommendation: Avenue 2

Avenue 1 risks a year-plus rebuilding what OpenRGB's GUI + Aura Creator already do, on Linux, less polished. Avenue 2 ships a different category of product in a niche where the commercial alternatives are vendor-locked or paradigm-different. Better fit for a solo + LLM-assisted developer.

What Avenue 2 requires from v1.2+

These graduate from "nice to have" to "required":

  1. Scheduler weights move to config. runtime.py:_scheduler_weight substring-matches "thermaltake", "140", "fractal" — ties scheduler behavior to Jack's specific zone names. An LLM can't generate working animations under that coupling. Weights become a per-zone field in runtime_config.json.
  2. Animation primitives become shape-agnostic. Builders currently hardcoded to specific LED counts (thermaltake_aurora.py = 1262 lines for 36 LEDs). Need to accept (zone_led_count, usable_indexes) so an LLM can write aurora_wave(zone_led_count=N) rather than hand-crafting arrays.
  3. The context pack is a deliverable. Shipped alongside rgb-aura: annotated source for runtime + writer + animation registry, primitive examples, OpenRGB API surface notes, zone shape conventions, canonical prompt. LLM-agnostic.
  4. Starter library of example animations. 6-12 well-crafted examples covering different effect families. LLMs generate better with examples to riff from than synthesizing from scratch.

What's NOT decided here

  • v1.2 issue list and milestone definition (separate planning step after this RFC resolves).
  • Whether a thin GUI/TUI eventually accompanies Avenue 2.
  • Which LLM(s) the canonical prompt targets. Pack should be LLM-agnostic; example prompts assume baseline capability.

Questions

  • Agree with Avenue 2?
  • Anything in the prior art that changes the calculus (i.e., is there a competitor strong enough to make this not worth pursuing)?
  • Once direction is locked: open a follow-up issue to add docs/project-direction.md + the three research files via PR, closing this RFC?
## Context v1.0 is shipped. v1.1.X scope is foundation work (bug fixes, test scaffolding, dead-file cleanup) and does not depend on this decision. **This RFC settles what rgb-aura aims to be in v1.2+.** Natural follow-on to `docs/llm-project-workflow.md`: that doc covered using an LLM to *build* rgb-aura. This RFC asks whether the next chapter should be a tool where the LLM helps *users* design their own RGB animations — same human-in-the-loop philosophy, one layer up. ## Two avenues considered **Avenue 1 — Full GUI Aura making studio.** Build a Linux equivalent of ASUS Aura Creator: timeline editor, preview canvas, profile manager, color pickers, hardware test loops. Familiar paradigm, large scope, year-plus of work, lands as "almost as good as OpenRGB GUI + Aura Creator clone." **Avenue 2 — LLM-assisted Aura making studio.** rgb-aura becomes the substrate for LLM-driven design. User describes the animation they want; an LLM produces working animation code using rgb-aura's API + a shipped context pack (primitives, examples, OpenRGB conventions, canonical prompt). Scope is bounded — the "studio" is the LLM. Plays to rgb-aura's existing strengths (safety architecture, modular animation registry, dry-run mode are exactly what an LLM needs to be productive). Multiplatform-friendly via OpenRGB + Python. ## Prior art (May 2026) Independently researched across three LLMs before writing this RFC. Full receipts in `docs/research/2026-05-10-prior-art-{perplexity,gemini,claude}.md`. Summary: - **Skydimo** ([skydimo.com](https://skydimo.com)) — proprietary, cross-vendor (1000+ brands), explicitly advertises *"describe your idea in natural language, AI instantly generates the corresponding lighting effects."* **Closest functional match.** Closed-source, not OpenRGB-based, AI embedded in their controller (not portable). - **Lumia Stream** ([lumiastream.com](https://lumiastream.com)) — commercial, ChatGPT-plugin-based natural-language RGB animation. Streaming-focused market. Same UX pattern, different segment. - **Corsair iCUE Murals** — commercial AI lighting, but **image/video-driven, not natural-language-driven.** Vendor-locked to Corsair. - **Razer Adaptive Immersive Experience** (GDC 2026) — runtime-adaptive lighting/haptics/audio. Not a design tool. - **Arduino Cloud AI Assistant** (Anthropic + Arduino, mid-2025) — *"describe it and Claude builds the sketch"* for microcontrollers. Closest pattern match for LLM + hardware + natural-language code generation. Vendor-integrated SaaS, single-LLM, different hardware target. - **NVIDIA Project G-Assist** — SLM controlling PC APIs including RGB as one of many actions. General PC assistant, not a design tool. - **TD-OpenRouter** (TouchDesigner, May 2026) — closest architectural analog of the context-pack pattern in a different creative domain. - **p5.js v2.0 community context packs** (HuggingFace, 2025+) — informal but established pattern of shipping API-doc + prompt bundles for LLM-assisted creative coding. **Honest novelty verdict: partially novel.** The high-level UX is no longer unprecedented — Skydimo and Lumia Stream have shipped commercial versions. Genuinely novel in implementation: 1. Open-source (vs. all named commercial competitors) 2. Built on OpenRGB (vs. proprietary protocol stacks) 3. **Portable context pack** users hand to any LLM, including local models (vs. AI embedded in vendor product — you can't take Skydimo's AI to Claude or a local Llama) 4. Linux-first (vs. Windows-first commercial offerings) 5. Natural-language-driven (vs. iCUE Murals image-driven) That's a defensible niche. ## Recommendation: Avenue 2 Avenue 1 risks a year-plus rebuilding what OpenRGB's GUI + Aura Creator already do, on Linux, less polished. Avenue 2 ships a different category of product in a niche where the commercial alternatives are vendor-locked or paradigm-different. Better fit for a solo + LLM-assisted developer. ## What Avenue 2 requires from v1.2+ These graduate from "nice to have" to "required": 1. **Scheduler weights move to config.** `runtime.py:_scheduler_weight` substring-matches `"thermaltake"`, `"140"`, `"fractal"` — ties scheduler behavior to Jack's specific zone names. An LLM can't generate working animations under that coupling. Weights become a per-zone field in `runtime_config.json`. 2. **Animation primitives become shape-agnostic.** Builders currently hardcoded to specific LED counts (`thermaltake_aurora.py` = 1262 lines for 36 LEDs). Need to accept `(zone_led_count, usable_indexes)` so an LLM can write `aurora_wave(zone_led_count=N)` rather than hand-crafting arrays. 3. **The context pack is a deliverable.** Shipped alongside rgb-aura: annotated source for runtime + writer + animation registry, primitive examples, OpenRGB API surface notes, zone shape conventions, canonical prompt. LLM-agnostic. 4. **Starter library of example animations.** 6-12 well-crafted examples covering different effect families. LLMs generate better with examples to riff from than synthesizing from scratch. ## What's NOT decided here - v1.2 issue list and milestone definition (separate planning step after this RFC resolves). - Whether a thin GUI/TUI eventually accompanies Avenue 2. - Which LLM(s) the canonical prompt targets. Pack should be LLM-agnostic; example prompts assume baseline capability. ## Questions - Agree with Avenue 2? - Anything in the prior art that changes the calculus (i.e., is there a competitor strong enough to make this not worth pursuing)? - Once direction is locked: open a follow-up issue to add `docs/project-direction.md` + the three research files via PR, closing this RFC?
JackFrostbyte commented 2026-05-11 02:27:45 -04:00
Owner
Copy link

Yeah i agree that avenue 2 is the way to go too.

Eventually a small gui just so user can easily setup their instance would be great.

But most people running linux should be confortable with some console work. We can easily keep that to a minimum and a semi interactive console setup.

The more moving parts, the more risk we have that something go wrong, and more maintenance to do. Do you want to start feathering the project? Or i can do it. LLM session to basically flesh out the project, what milestone we need to create, and create all the issue that will be coded and tested one by one.

The first thing we need to do is iron out the already present issue, clean up the file that are there that do nothing that Claude already suggested. If you want to do that go for it. Ask claude to respect a rigorous GIT workflow so that every modification is traceable. For the bugs themselves, you can create an issue, and a branch for tha issue. When everything is tested we will merge and proceed to the next. I dont know if you can merge with you user status on the project.

Yeah i agree that avenue 2 is the way to go too. Eventually a small gui just so user can easily setup their instance would be great. But most people running linux should be confortable with some console work. We can easily keep that to a minimum and a semi interactive console setup. The more moving parts, the more risk we have that something go wrong, and more maintenance to do. Do you want to start feathering the project? Or i can do it. LLM session to basically flesh out the project, what milestone we need to create, and create all the issue that will be coded and tested one by one. The first thing we need to do is iron out the already present issue, clean up the file that are there that do nothing that Claude already suggested. If you want to do that go for it. Ask claude to respect a rigorous GIT workflow so that every modification is traceable. For the bugs themselves, you can create an issue, and a branch for tha issue. When everything is tested we will merge and proceed to the next. I dont know if you can merge with you user status on the project.
McJuniorstein commented 2026-05-11 03:10:01 -04:00
Author
Collaborator
Copy link

Confirmed I have write access on the repo — checked via API, write permission, can push, branch, and merge PRs. So we're good on that side.

Taking you up on the v1.1.X cycle and leaving v1.2 planning to your LLM session — your project, your fingerprints on direction belong there. Claude handles the v1.1.X work, me reviewing along the way, you merging.

Heads up on the asymmetry: Claude has direct repo access via API on my end, so issues, branches, and PRs all happen straight from this side. You're driving GPT through copy/paste in the browser, so we'll keep PRs small, well-described, and easy for you to read and merge in batches. This is my first time collaborating on someone else's repo, so I'm erring careful — small atomic PRs, nothing touches main directly, every change tied to an issue, nothing self-merged.

One workflow note: I'd like to keep project conversations here on the repo rather than Discord where possible. Things get missed when discussion splits across platforms, and the repo gives us a permanent traceable record. Discord stays fine for off-topic / quick stuff.

Plan:

  1. Follow-up to this RFC (#43): open a separate issue + branch to add docs/project-direction.md and the three research files (docs/research/2026-05-10-prior-art-{perplexity,gemini,claude}.md) documenting the v1.2+ direction decision. PR closes #43.
  2. v1.1 milestone to group the bug-swatting cycle.
  3. All v1.1.X issues opened at once so you can see scope before any code lands, push back if anything's out of scope.
  4. Start with the easiest one (dead-file cleanup) as a workflow test — if you like how the PR reads and merges, we batch the rest.

v1.1.X work I'm proposing, based on Claude's earlier code review:

  • runtime.py:825 UnboundLocalError fix + regression test
  • Delete the five 0-byte skeleton files
  • Add SIGHUP to the signal handler
  • tests/test_device_mapping.py + tests/conftest.py — populate the empty test dir
  • leave_dark_blue default vs docs/safe-failure-modes.md mismatch — your call on flip-default vs amend-doc
  • Dry-run wait-line collapsing UX cleanup
  • Animation frame integrity tests (parametrized over the registry)

Flag anything out of scope for v1.1.X before we start. Sound good?

Confirmed I have write access on the repo — checked via API, `write` permission, can push, branch, and merge PRs. So we're good on that side. Taking you up on the v1.1.X cycle and leaving v1.2 planning to your LLM session — your project, your fingerprints on direction belong there. Claude handles the v1.1.X work, me reviewing along the way, you merging. Heads up on the asymmetry: Claude has direct repo access via API on my end, so issues, branches, and PRs all happen straight from this side. You're driving GPT through copy/paste in the browser, so we'll keep PRs small, well-described, and easy for you to read and merge in batches. This is my first time collaborating on someone else's repo, so I'm erring careful — small atomic PRs, nothing touches main directly, every change tied to an issue, nothing self-merged. One workflow note: I'd like to keep project conversations here on the repo rather than Discord where possible. Things get missed when discussion splits across platforms, and the repo gives us a permanent traceable record. Discord stays fine for off-topic / quick stuff. Plan: 1. **Follow-up to this RFC (#43):** open a separate issue + branch to add `docs/project-direction.md` and the three research files (`docs/research/2026-05-10-prior-art-{perplexity,gemini,claude}.md`) documenting the v1.2+ direction decision. PR closes #43. 2. **v1.1 milestone** to group the bug-swatting cycle. 3. **All v1.1.X issues opened at once** so you can see scope before any code lands, push back if anything's out of scope. 4. **Start with the easiest one** (dead-file cleanup) as a workflow test — if you like how the PR reads and merges, we batch the rest. v1.1.X work I'm proposing, based on Claude's earlier code review: - `runtime.py:825` `UnboundLocalError` fix + regression test - Delete the five 0-byte skeleton files - Add SIGHUP to the signal handler - `tests/test_device_mapping.py` + `tests/conftest.py` — populate the empty test dir - `leave_dark_blue` default vs `docs/safe-failure-modes.md` mismatch — your call on flip-default vs amend-doc - Dry-run wait-line collapsing UX cleanup - Animation frame integrity tests (parametrized over the registry) Flag anything out of scope for v1.1.X before we start. Sound good?
JackFrostbyte commented 2026-05-11 20:27:41 -04:00
Owner
Copy link

Yeah thats perfect.

I agree with everything.

I have discovered another other bug to solve while using it. It almost never start correctly from the service. Openrgb seem to never discover my GPU until i open it and rescan the devices. Only then the service start on it's next retry. So in the 1.0.X branch we we will start, you can do the file cleanup as 1.0.1, then issue 1.0.2 would be the rest of the bugs previously mentionned. 1.0.3 would be to find a way to make the SDK auto re-scan devices on startup before starting the aura service itself. When the bugs will be dealt with, 1.1.X branch would be the start of the developement of the avenue already discussed, that will culminate in version 1.2 that will be the first stable version we have of the working improvement of the LLM assisted aura creation tool. This issue itself will have no branch for it, it is only used to set the plan for the future.

I also never worked in cooperation on a project yet. It is part of why i wanted to start using repo so the experience will benefit us both for sure.

Yeah thats perfect. I agree with everything. I have discovered another other bug to solve while using it. It almost never start correctly from the service. Openrgb seem to never discover my GPU until i open it and rescan the devices. Only then the service start on it's next retry. So in the 1.0.X branch we we will start, you can do the file cleanup as 1.0.1, then issue 1.0.2 would be the rest of the bugs previously mentionned. 1.0.3 would be to find a way to make the SDK auto re-scan devices on startup before starting the aura service itself. When the bugs will be dealt with, 1.1.X branch would be the start of the developement of the avenue already discussed, that will culminate in version 1.2 that will be the first stable version we have of the working improvement of the LLM assisted aura creation tool. This issue itself will have no branch for it, it is only used to set the plan for the future. I also never worked in cooperation on a project yet. It is part of why i wanted to start using repo so the experience will benefit us both for sure.
JackFrostbyte commented 2026-05-12 00:40:35 -04:00
Owner
Copy link

I just asked GPT about the collaborative workflow and proper branching of the repo and this is the answer :

I think we should use a slightly more formal branch structure now that collaboration is starting:

  • main stays stable and only receives tested release-ready code.
  • develop becomes the integration/testing branch.
  • Each issue gets its own small branch from develop.
  • PRs merge back into develop after review/testing.
  • When enough work is accumulated for a release, develop is merged into main, then tagged.
  • No direct commits to main.
  • No self-merge for collaborator PRs unless explicitly agreed for trivial documentation-only changes.

For urgent fixes to the current stable release, we can branch from main as hotfix/..., merge back into main, tag the patch release, then merge/cherry-pick that fix back into develop.

This should keep main clean while still allowing us to test several small PRs before deciding when a new tag is justified.

I just asked GPT about the collaborative workflow and proper branching of the repo and this is the answer : I think we should use a slightly more formal branch structure now that collaboration is starting: - `main` stays stable and only receives tested release-ready code. - `develop` becomes the integration/testing branch. - Each issue gets its own small branch from `develop`. - PRs merge back into `develop` after review/testing. - When enough work is accumulated for a release, `develop` is merged into `main`, then tagged. - No direct commits to `main`. - No self-merge for collaborator PRs unless explicitly agreed for trivial documentation-only changes. For urgent fixes to the current stable release, we can branch from `main` as `hotfix/...`, merge back into `main`, tag the patch release, then merge/cherry-pick that fix back into `develop`. This should keep `main` clean while still allowing us to test several small PRs before deciding when a new tag is justified.
JackFrostbyte commented 2026-05-15 18:10:01 -04:00
Owner
Copy link

Closing this RFC as resolved.

The project direction for v1.2+ is now chosen: RGB Aura will move toward Avenue 2, the LLM-assisted Aura making studio approach, rather than trying to become a full traditional GUI/Aura Creator clone.

This issue served its purpose as a direction-setting discussion. The decision is documented in the conversation, the development path has been agreed on, and several practical steps have already been put in place toward that goal:

  • v1.0.x is being used to clean up the current foundation and fix known runtime/service issues.
  • v1.1.x will begin the structural work needed for the LLM-assisted workflow.
  • v1.2 is the target for the first stable version of the project shaped around that new direction.
  • The project will keep the human-in-the-loop, LLM-assisted development philosophy, but move it one layer higher: helping users design their own RGB animations through a portable context/prompt/code-generation workflow.

No branch is required for this RFC itself. Follow-up implementation work will continue through dedicated issues and PRs.

Resolved as a planning/direction issue.

Closing this RFC as resolved. The project direction for v1.2+ is now chosen: RGB Aura will move toward Avenue 2, the LLM-assisted Aura making studio approach, rather than trying to become a full traditional GUI/Aura Creator clone. This issue served its purpose as a direction-setting discussion. The decision is documented in the conversation, the development path has been agreed on, and several practical steps have already been put in place toward that goal: * v1.0.x is being used to clean up the current foundation and fix known runtime/service issues. * v1.1.x will begin the structural work needed for the LLM-assisted workflow. * v1.2 is the target for the first stable version of the project shaped around that new direction. * The project will keep the human-in-the-loop, LLM-assisted development philosophy, but move it one layer higher: helping users design their own RGB animations through a portable context/prompt/code-generation workflow. No branch is required for this RFC itself. Follow-up implementation work will continue through dedicated issues and PRs. Resolved as a planning/direction issue.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
docs/issue-72-animation-pack-architecture
develop
v1.0.1
v1.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
JackFrostbyte/rgb-aura#43
No description provided.