DESCRIPTION

A quark_queue is the main runtime datastructure of quark, it is loosely called a queue as it's where events will originate from. Events will be collected into the queue, buffered, aggregated and filtered if necessary.

The quark_queue_open function does the following:

• Attempts to use the selected backend. This includes loading the EBPF programs for EBPF or the probes for KPROBES. Exactly one backend must be selected, anything else fails with EINVAL. There is no fallback: users that want one must retry with another backend themselves if quark_queue_open fails.
• On its first call it will also initialize global host state, like BTF offsets and HZ.
• Initializes the various lists and internal buffers of qq.
• If KPROBES is selected, it initializes one perf-ring per-cpu in order to collect process events, see quark_queue_get_epollfd(3) and quark_queue_block(3).
• If EBPF is selected, it initializes an EBPF ringbuffer, support for old style perf-rings with EBPF is currently not supported.
• Scrapes /proc for a snapshot of the existing processes in the system. quark_queue_open is smart enough to open the rings before the scraping, as to be make sure no process is lost. These initial processes are available through quark_process_iter(3).

Default queue behaviour can be tweaked with attr. A default configuration for tweaking can be acquired via quark_queue_default_attr(3). In case attr is NULL, the default configuration is used.

struct quark_queue_attr is defined as:

struct quark_queue_attr {
	int	 flags;
	int	 max_length;
	int	 cache_grace_time;	/* in milliseconds */
	int	 hold_time;		/* in milliseconds */
	...
};

flags

Bitmask of:

QQ_EBPF

Use the EBPF backend, this is the default of quark_queue_default_attr(3).

QQ_KPROBE

Use the KPROBE backend. Event times are CLOCK_MONOTONIC samples translated to CLOCK_BOOTTIME, since perf can not stamp them with CLOCK_BOOTTIME directly. The translation is exact, with one exception: events stamped before a suspend but still unread when the host resumes are translated based on resume time and shift late by up to that suspend's duration. Only events in flight across the suspend are affected, hosts that never suspend are always exact.

QQ_THREAD_EVENTS

Include per-thread events, instead of per-process events. This option will be removed in the future, but it may be useful for debugging.

QQ_MIN_AGG

Don't aggregate fork, exec and exit, perform only minimal aggregation.

QQ_ENTRY_LEADER

Include proc_entry_leader and proc_entry_type in quark_events. Entry leader is how the process entered the system, it is disabled by default as it is Elastic/ECS specific.

max_length

The maximum size of the internal buffering queue in number of events.

Quark buffers each event for a computed interval in order to sort and aggregate multiple events into one. The closer the queue is to being full, the smaller the interval: until quark decides to not buffer events at all.

cache_grace_time

The grace period for removing an event from the cache.

When a process exits, it is removed from the cache, but only after cache_grace_time, this gives the user a small window where it can still query a terminated process via quark_process_lookup(3).

hold_time

How long to buffer (hold) an event before delivering it to the user via quark_queue_get_event(3).

Events received from the backend are not immediately forwarded to the user, this allows multiple events to be aggregated as well as ordered by time. In case quark is overloaded, it will use a stepping function where hold_time decreases the more loaded it is.

Details are described in quark(7).