DESCRIPTION

A quark_queue is the main runtime datastructure of quark, it is loosely called a queue as it's where events will originate from. Events will be collected into the queue, buffered, aggregated and filtered if necessary.

The quark_queue_open function does the following:

• Attempts to use the best backend available unless otherwise especified. This includes loading the EBPF programs for EBPF or the probes for KPROBES. Only one backend is used and it defaults to EBPF and falls back to KPROBE.
• On its first call it will also initialize global host state, like BTF offsets and HZ.
• Initializes the various lists and internal buffers of qq.
• If KPROBES is selected, it initializes one perf-ring per-cpu in order to collect process events, see quark_queue_get_epollfd(3) and quark_queue_block(3).
• If EBPF is selected, it initializes an EBPF ringbuffer, support for old style perf-rings with EBPF is currently not supported.
• Scrapes /proc for a snapshot of the existing processes in the system. quark_queue_open is smart enough to open the rings before the scraping, as to be make sure no process is lost. These initial process events will be available on the first call to quark_queue_get_event(3) with the events member set to QUARK_EV_SNAPSHOT.

Default queue behaviour can be tweaked with attr. A default configuration for tweaking can be acquired via quark_queue_default_attr(3). In case attr is NULL, the default configuration is used.

struct quark_queue_attr is defined as:

struct quark_queue_attr {
	int	 flags;
	int	 max_length;
	int	 cache_grace_time;	/* in milliseconds */
	int	 hold_time;		/* in milliseconds */
	...
};

flags

Bitmask of:

QQ_EBPF

Enable the EBPF backend. EBPF is attempted first and falls back to KPROBE if both were specified.

QQ_KPROBE

Enable the KPROBE backend, see above.

QQ_ALL_BACKENDS

Shorthand for (QQ_EBPF | QQ_KPROBE).

QQ_THREAD_EVENTS

Include per-thread events, instead of per-process events. This option will be removed in the future, but it may be useful for debugging.

QQ_NO_SNAPSHOT

Don't send the initial snapshot of existing processes.

QQ_MIN_AGG

Don't aggregate fork, exec and exit, perform only minimal aggregation.

QQ_ENTRY_LEADER

Include proc_entry_leader and proc_entry_type in quark_events. Entry leader is how the process entered the system, it is disabled by default as it is Elastic/ECS specific.

max_length

The maximum size of the internal buffering queue in number of events.

Quark buffers each event for a computed interval in order to sort and aggregate multiple events into one. The closer the queue is to being full, the smaller the interval: until quark decides to not buffer events at all.

cache_grace_time

The grace period for removing an event from the cache.

When a process exits, it is removed from the cache, but only after cache_grace_time, this gives the user a small window where it can still query a terminated process via quark_process_lookup(3).

hold_time

How long to buffer (hold) an event before delivering it to the user via quark_queue_get_event(3).

Events received from the backend are not immediately forwarded to the user, this allows multiple events to be aggregated as well as ordered by time. In case quark is overloaded, it will use a stepping function where hold_time decreases the more loaded it is.

Details are described in quark(7).