[DRAFT/TESTING] userspace LL/audio test PR (latest ver: V42) - #10558
Open
kv2019i wants to merge 16 commits into
Open
[DRAFT/TESTING] userspace LL/audio test PR (latest ver: V42)#10558kv2019i wants to merge 16 commits into
kv2019i wants to merge 16 commits into
Conversation
Collaborator
Author
|
Running the pipeline_two_components_user test added in this PR, on a Intel PTL, looks like this: |
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
March 9, 2026 17:07
9cdb1be to
37b0412
Compare
Collaborator
Author
|
V2 snapshot pushed:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
March 17, 2026 18:29
37b0412 to
7317b18
Compare
Collaborator
Author
|
V3 snapshot pushed:
|
Collaborator
Author
|
Test output for the new test added in V3: |
kv2019i
commented
Mar 17, 2026
| /* works? yes */ | ||
| //return 0; | ||
|
|
||
| printk("ipc %p\n", ipc); |
Collaborator
Author
There was a problem hiding this comment.
Oops, these shouldn't be here. :)
kv2019i
commented
Mar 17, 2026
|
|
||
| /* create the pipeline */ | ||
| pipe = pipeline_new(NULL, pipe_desc->primary.r.instance_id, | ||
| pipe = pipeline_new(heap, pipe_desc->primary.r.instance_id, |
Collaborator
Author
There was a problem hiding this comment.
@lyakh @jsarha @lgirdwood This is where you'd plug in the vregions stuff to pass a separate heap to each pipe (based on topology description of its needs). In this series, as a placeholder, I use the zephyr_ll_user_heap() instead.
Collaborator
There was a problem hiding this comment.
right, maybe put a comment there for now to make re-discovery easier
lyakh
reviewed
Mar 18, 2026
lyakh
left a comment
lyakh
left a comment
Collaborator
There was a problem hiding this comment.
last reviewed commit so far "schedule: zephyr_ll: implement thread_init/free domain ops"
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| void comp_grant_access_to_thread(const struct comp_dev *dev, struct k_thread *th) | ||
| { | ||
| assert(dev->list_mutex); |
Collaborator
There was a problem hiding this comment.
description a bit confusing - this is only granting access to a mutex. Also list_mutex is only added to comp_dev in the next commit.
| } | ||
|
|
||
| stream_addr = rballoc_align(flags, size, align); | ||
| stream_addr = sof_heap_alloc(heap, flags, size, align); |
Collaborator
There was a problem hiding this comment.
the commit, that is mentioned in the commit message, only moved buffer context objects to particular heaps. This commit moves actual data buffers to them too, which is different and (arguably) more risky
| #define HDA_DMA_BUFFER_PERIOD_COUNT 4 | ||
|
|
||
| SHARED_DATA struct sof_dma dma[] = { | ||
| APP_TASK_DATA SHARED_DATA struct sof_dma dma[] = { |
Collaborator
There was a problem hiding this comment.
do I understand correctly, that this kind of userspace access makes that data writable to userspace?
| comp_err(dev, "requested channel %d is busy", hda_chan); | ||
| return -ENODEV; | ||
| } | ||
| hd->chan = &hd->dma->chan[channel]; |
Collaborator
There was a problem hiding this comment.
is this also not needed for the legacy mode? Also below
| uint64_t next_sync; | ||
| uint64_t period_in_cycles; | ||
| #endif | ||
| struct k_heap *heap; |
Collaborator
There was a problem hiding this comment.
wasn't this already referenced in the previous commit?
|
|
||
| k_spinlock_init(&dd->dai->lock); | ||
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| dd->dai->lock = k_object_alloc(K_OBJ_MUTEX); |
Collaborator
There was a problem hiding this comment.
check for NULL? Possibly in other locations too
| k_mutex_lock(dai->lock, K_FOREVER); | ||
| props = dai_get_properties(dai->dev, direction, stream_id); | ||
| hs_id = props->dma_hs_id; | ||
| ret = dai_get_properties_copy(dai->dev, direction, stream_id, &props); |
Collaborator
There was a problem hiding this comment.
I'm guessing this is made a syscall in one of the commits
| mod_heap = &mod_heap_user->heap; | ||
| } else { | ||
| #ifdef CONFIG_SOF_USERSPACE_LL | ||
| mod_heap = zephyr_ll_user_heap(); |
Collaborator
There was a problem hiding this comment.
looks good, but this else is entered under multiple conditions, might need to double-check
| .schedule_task_before = zephyr_ll_task_schedule_before, | ||
| .schedule_task_after = zephyr_ll_task_schedule_after, | ||
| .schedule_task_free = zephyr_ll_task_free, | ||
| .schedule_task_free = zephyr_ll_task_sched_free, |
Collaborator
There was a problem hiding this comment.
let's "spend" 3 more characters and make it ..._schedule_free()
| return -ENOMEM; | ||
| } | ||
| tr_err(&ll_tr, "Failed to allocate thread object for core %d", core); | ||
| dt->handler = NULL; |
lyakh
reviewed
Mar 18, 2026
| { | ||
| const struct sof_man_fw_desc *desc = basefw_vendor_get_manifest(); | ||
| const struct sof_man_module *mod; | ||
| uint32_t i; |
| uint32_t i; | ||
|
|
||
| if (!desc) | ||
| return -1; |
| return (int)i; | ||
| } | ||
|
|
||
| return -1; |
| union ipc4_connector_node_id node_id; | ||
| uint32_t dma_buffer_size; | ||
| uint32_t config_length; | ||
| } __packed __aligned(4); |
Collaborator
There was a problem hiding this comment.
does __aligned actually make sense in a type definition?
| pipe_msg.extension.dat = ipc_user->ipc_msg_ext; | ||
|
|
||
| /* Execute pipeline creation in user context */ | ||
| ipc_user->result = ipc_pipeline_new(ipc_user->ipc, (ipc_pipe_new *)&pipe_msg); |
Collaborator
There was a problem hiding this comment.
that's brave! ;-) I'd put a huge "TODO" here to make sure not to ship this by chance :-)
|
|
||
| /* create the pipeline */ | ||
| pipe = pipeline_new(NULL, pipe_desc->primary.r.instance_id, | ||
| pipe = pipeline_new(heap, pipe_desc->primary.r.instance_id, |
Collaborator
There was a problem hiding this comment.
right, maybe put a comment there for now to make re-discovery easier
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
March 27, 2026 13:37
7317b18 to
c029d49
Compare
Collaborator
Author
|
V4 snapshot pushed:
|
Collaborator
Author
|
Example output with V4 patchset: |
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
March 30, 2026 17:09
c029d49 to
9c778e4
Compare
Collaborator
Author
|
V5 pushed:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
April 1, 2026 13:06
9c778e4 to
c4b127f
Compare
Collaborator
Author
|
V6 pushed:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
April 1, 2026 14:36
c4b127f to
ad359d6
Compare
Collaborator
Author
|
V7 submitted:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
April 10, 2026 11:48
ad359d6 to
c7e9a37
Compare
Collaborator
Author
|
V8 submitted:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
April 10, 2026 12:04
c7e9a37 to
8af1be4
Compare
Collaborator
Author
|
V9 submitted:
|
Collaborator
Author
|
V10:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
April 15, 2026 11:58
aa5efe0 to
35a61b6
Compare
Collaborator
Author
|
V11 pushed:
|
kv2019i
force-pushed
the
feat/userspace-ll-wip
branch
from
April 15, 2026 13:03
35a61b6 to
214c051
Compare
Collaborator
Author
|
V12 pushed:
|
Collaborator
Author
|
V36:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR is an early-stage step toward running low-latency (LL) audio pipeline work in Zephyr user-space by making pipeline position tracking safe/accessible from user threads, and by enabling the required Zephyr userspace configuration on ACE30 boards.
Changes:
- Move
pipeline_posn_init()earlier in the Zephyr init sequence and add user-thread access grants for the pipeline position mutex. - Update pipeline position locking to use a mutex (userspace-safe) when
CONFIG_SOF_USERSPACE_LLis enabled, and avoid copying non-userspace-accessible trace context. - Enable/adjust userspace LL-related Kconfig settings in ACE30 PTL/WCL board configs and disable features noted as not yet userspace-compatible.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| zephyr/wrapper.c | Removes late pipeline position init from task_main_start() (init is moved earlier). |
| src/schedule/zephyr_ll.c | Adds warning/handling when task-free semaphore wait times out. |
| src/ipc/ipc-common.c | Grants IPC userspace thread access to the pipeline position mutex. |
| src/init/init.c | Calls pipeline_posn_init() earlier (before platform_init() / userspace IPC init). |
| src/include/sof/audio/pipeline.h | Declares pipeline_posn_grant_access() for userspace LL builds. |
| src/include/sof/audio/pipeline-trace.h | Adds a build-time guard for userspace LL + non-Zephyr-log trace usage. |
| src/audio/pipeline/pipeline-graph.c | Uses mutex-based locking for pipeline position offsets under userspace LL and adds access-grant helper. |
| app/boards/intel_adsp_ace30_wcl.conf | Enables userspace + userspace LL and disables currently incompatible features. |
| app/boards/intel_adsp_ace30_ptl.conf | Enables userspace LL and disables currently incompatible features. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+626
to
+630
| if (wait_ret && task->state != SOF_TASK_STATE_FREE) { | ||
| tr_warn(&ll_tr, "task %p still active on free (state %d, semret %d)", | ||
| task, task->state, wait_ret); | ||
| zephyr_ll_task_done(sch, task); | ||
| } |
Collaborator
Author
|
V37 version:
|
Collaborator
Author
|
V38 update:
|
Collaborator
Author
|
V38b:
|
PR 10558: test resultsRun date: 2026-09-03 14:10 UTC Tested commit: af2794e0dad576b1fba2df5bf23e580562aef6c0
|
Collaborator
Author
|
V39 update:
|
Add a check to ensure task state is what is expected after k_sem_take() returns in zephyr_ll_task_free(). This is needed to avoid a rare error hit when running stress tests with chain DMA in user-space LL builds. Issue is hard to reproduce, but similar error signature can be created by passing K_NO_WAIT to k_sem_take() and running a test with chain-dma pipeline. Add defensive code that handles this scenario and prints out a warning when unexpected return occurs. Tested with a custom build with K_NO_WAIT passed to k_sem_take(). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The direction-of-arrival emphasis coefficient tables iir_emphasis_48k and iir_emphasis_16k were declared as mutable globals, so the linker placed them in the .data section. The tables are read-only coefficient data, never modified, so mark them static const. This mirrors how sound_dose stores its IIR coefficient table. As another benefit, this change makes it possible to run tdfb in user-space as user threads can access rodata. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Extract a privileged LLEXT-related part from lib_manager_module_create() into a separate function to be called from kernel context. At the same time lib_manager_mod_free_priv() already executes privileged operations; to make it callable in userspace, convert lib_manager_free_module() to a system call. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 3a21e7d)
If LL runs in userspace, it needs access to loaded LLEXT modules, running in DP more too. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 95693d1)
Make scheduling LL thread and synchronisation objects per-core and forward IPCs and scheduling events accordingly. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 17cab14)
Checking pointer for non-NULL after dereferencing it makes no sense. Swap the order. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit eedc4a1)
In case of userspace LL scheduling the (also userspace) IPC thread needs access rights to DP assets like the thread itself and its stack and synchronisation primitives. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 9c61c68)
When running in syscall context on behalf of a userspace thread dynamically mapped memory doesn't automatically become accessible. To make it accessible it has to be added to the thread memory domain. This is a problem for loadable modules with executable cold sections. To be able to execute them they have to be mapped to threads with the executable bit set. While for linking that memory has to be mapped writable. To solve the problem we perform linking from the kernel IPC context before forwarding to the userspace IPC thread. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 4e11621)
Collaborator
Author
|
V40 update:
|
Collaborator
Author
|
V41 update:
|
Re-enable LLEXT, DRAM execution and data, the DP scheduler and boot time testing with userspace LL with a small restriction for the latter - userspace allocation failure tests aren't currently working. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com> (cherry picked from commit 4f17b68)
z_vrfy_ipc_msg_list_remove() rejected any message that was not currently on ipc->msg_list by failing K_SYSCALL_VERIFY(found), which turns into a kernel oops. But ipc_msg_list_remove() is called from ipc_msg_free() / mod_ipc_msg_free() to drop a message that may or may not still be queued. The common case at stream stop / pipeline delete is freeing a message that has already been sent and dequeued: its list node is self-linked (empty), so it is not "found" and the verifier oopses the LL user thread with: <err> os.z_vrfy_ipc_msg_list_remove: syscall z_vrfy_ipc_msg_list_remove ... failed check: found <err> os.z_fatal_error: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0 Relax the checks to avoid this scenario. If the msg->list is empty, it is safe to call z_impl_ipc_msg_list_remove(). The msg->list pointer itself is already verified with K_SYSCALL_MEMORY_WRITE(). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
…sent
llext_manager_rm_mod_domain() removes the .cold and .coldrodata memory
partitions of a module from its domain if the corresponding
mctx->segment[] descriptor has a non-zero address. Those descriptors are
only ever populated by llext_manager_add_mod_domain() when the module
actually contains a .cold / .coldrodata section.
The module context array is allocated with rmalloc() (non-zeroing) and
llext_manager_mod_init() initialises every field except the segment[]
array. The LIB_MANAGER_TEXT/RODATA/DATA/BSS descriptors are filled in
during linking, but LIB_MANAGER_COLD / LIB_MANAGER_COLDRODATA are left
holding uninitialised heap data for any module that has no such section.
As a result, when such a module is freed, rm_mod_domain() sees a garbage
non-zero address and tries to remove a partition that was never added:
<err> os.k_mem_domain_remove_partition: no matching partition found
<err> lib_manager.llext_manager_rm_mod_domain: failed to remove
.coldrodata memory partition: -2
Reproduced at end-of-stream of a DMIC capture (arecord -Dhw:0,4), whose
EQIIR / TDFB / DRC modules carry no .coldrodata section.
Make add_mod_domain() the sole owner of these descriptors: explicitly
zero them when the section is absent, so rm_mod_domain() only ever
removes partitions that were really added.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
…aths vregion_verify() asserts that the vregion metadata object is NOT accessible to the userspace context. It did this with K_OOPS(!K_SYSCALL_MEMORY_READ(vr, sizeof(*vr))); but K_SYSCALL_MEMORY_READ() emits an "os.vregion_verify: ... Memory region <addr> (size 88) read access denied" error via LOG_ERR precisely when the region is inaccessible - i.e. in the expected, correct case for a kernel-only vregion. Probe the mapping directly with arch_buffer_validate(), which performs the same check without logging, and oops only if the userspace context can actually read the metadata. No functional change to the security check; only the false-positive error logging is removed. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Bump the LL heap size of 0x8000 (512 KiB). This is current maximum with CONFIG_VIRTUAL_HEAP (enabled by default for PTL). Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Make the options from app/overlays/ptl/ll_userspace_overlay.conf the default for the Intel Panther Lake (ptl) and Wildcat Lake (wcl) build targets, so user-space Low-Latency audio pipelines are enabled without having to pass the overlay explicitly. As noted in the overlay header, once user-space LL is enabled for a target by default the settings belong in the SOF board file directly. For ptl the board already provides the user-space base (USERSPACE, dynamic threads, MMU L2 tables, domain partitions), so only the LL overlay options are added and the conflicting telemetry / cold-store / llext / modules defaults are flipped to match the overlay. wcl had no user-space base at all; since CONFIG_SOF_USERSPACE_LL depends on CONFIG_USERSPACE it would otherwise be silently dropped. Mirror ptl's user-space base into the wcl board file as well so LL actually takes effect there. The ll_userspace_overlay.conf file is kept unchanged; it now re-applies identical values and remains usable by development build scripts. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Collaborator
Author
|
V42 update:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SOF has recently gained ability to run DP (=preemptable audio tasks) in Zephyr user-space.
This PR is an early stage pull-request for changes to extend this capability to all of the audio pipeline code, and specifically the LL (low-latency) tasks.
This early stage as the design is not set in stone and the PR uses a number of short cuts in order to move (and tests) incrementally larger sets of audio functionality.