File: //usr/share/doc/rsyslog/html/rainerscript/queue_parameters.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>General Queue Parameters — rsyslog 8.2006.0 documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/rsyslog.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '8.2006.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="The rsyslog “call” statement" href="rainerscript_call.html" />
<link rel="prev" title="Lookup Tables" href="lookup_tables.html" />
</head>
<body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="rainerscript_call.html" title="The rsyslog “call” statement"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="lookup_tables.html" title="Lookup Tables"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">rsyslog 8.2006.0 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../configuration/index.html" >Configuration</a> »</li>
<li class="nav-item nav-item-2"><a href="index.html" accesskey="U">RainerScript</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="general-queue-parameters">
<h1>General Queue Parameters<a class="headerlink" href="#general-queue-parameters" title="Permalink to this headline">¶</a></h1>
<table border="1" class="docutils">
<colgroup>
<col width="26%" />
<col width="74%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><strong>Authors:</strong></td>
<td><a class="reference external" href="https://rainer.gerhards.net/">Rainer Gerhards</a> <<a class="reference external" href="mailto:rgerhards%40adiscon.com">rgerhards<span>@</span>adiscon<span>.</span>com</a>>;</td>
</tr>
</tbody>
</table>
<div class="section" id="usage">
<h2>Usage<a class="headerlink" href="#usage" title="Permalink to this headline">¶</a></h2>
<p>Queue parameters can be used together with the following statements:</p>
<ul class="simple">
<li><a class="reference internal" href="../configuration/actions.html"><span class="doc">action()</span></a></li>
<li>ruleset()</li>
<li>main_queue()</li>
</ul>
<p>Queues need to be configured in the action or ruleset it should affect.
If nothing is configured, default values will be used. Thus, the default
ruleset has only the default main queue. Specific Action queues are not
set up by default.</p>
<p>To fully understand queue parameters and how they interact, be sure to
read the <a class="reference internal" href="../concepts/queues.html"><span class="doc">queues</span></a> documentation.</p>
</div>
<div class="section" id="configuration-parameters">
<h2>Configuration Parameters<a class="headerlink" href="#configuration-parameters" title="Permalink to this headline">¶</a></h2>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">As with other configuration objects, parameters for this
object are case-insensitive.</p>
</div>
<div class="section" id="queue-filename">
<h3>queue.filename<a class="headerlink" href="#queue-filename" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>word</td>
<td>none</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueFileName</span></code></td>
</tr>
</tbody>
</table>
<p>File name to be used for the queue files. If specified, this parameter
enables disk-assisted queue functionality. If <em>not</em> specified,
the queue will operate without saving the queue to disk, either
during its operation or when shut down. See the separate
<code class="docutils literal"><span class="pre">queue.saveonshutdown</span></code> parameter to configure that option.
Please note that this is actually just the file name. A directory
can NOT be specified in this parameter. If the files shall be
created in a specific directory, specify <code class="docutils literal"><span class="pre">queue.spoolDirectory</span></code>
for this. The filename is used to build to complete path for queue
files.</p>
</div>
<div class="section" id="queue-spooldirectory">
<h3>queue.spoolDirectory<a class="headerlink" href="#queue-spooldirectory" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>word</td>
<td>none</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<p>This is the directory into which queue files will be stored. Note
that the directory must exist, it is NOT automatically created by
rsyslog. If no spoolDirectory is specified, the work directory is
used.</p>
</div>
<div class="section" id="queue-size">
<h3>queue.size<a class="headerlink" href="#queue-size" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>1000/50000</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueSize</span></code></td>
</tr>
</tbody>
</table>
<p>This is the maximum size of the queue in number of messages. Note
that setting the queue size to very small values (roughly below 100
messages) is not supported and can lead to unpredictable results.
For more information on the current status of this restriction see
the <a class="reference external" href="http://www.rsyslog.com/lower-bound-for-queue-sizes/">rsyslog FAQ: “lower bound for queue
sizes”</a>.</p>
<p>The default depends on queue type and rsyslog version, if you need
a specific value, please specify it. Otherwise rsyslog selects what
it considers appropriate for the version in question. In rsyslog
rsyslog 8.30.0, for example, ruleset queues have a default size
of 50000 and action queues which are configured to be non-direct
have a size of 1000.</p>
</div>
<div class="section" id="queue-dequeuebatchsize">
<h3>queue.dequeueBatchSize<a class="headerlink" href="#queue-dequeuebatchsize" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>128/1024</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueDequeueBatchSize</span></code></td>
</tr>
</tbody>
</table>
<p>Specifies the maximum batch size for dequeue operations. This setting affects performance.
As a rule of thumb, larger batch sizes (up to a environment-induced upper limit)
provide better performance. For the average system, there usually should be no need
to adjust batch sizes as the defaults are sufficient. The default for ruleset queues
is 1024 and for action queues 128.</p>
<p>Note that this only specifies the <strong>maximum</strong> batch size. Batches will be slower if
rsyslog does not have as many messages inside the queue at time of dequeuing it.
If you want to set a minimum Batch size as well, you can use <cite>queue.minDequeueBatchSize</cite>.</p>
</div>
<div class="section" id="queue-mindequeuebatchsize">
<h3>queue.minDequeueBatchSize<a class="headerlink" href="#queue-mindequeuebatchsize" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>0</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<p>Specifies the <strong>minimum</strong> batch size for dequeue operations. This setting is especially
useful with outputs like ElasticSearch or ClickHouse, where you want to limit the
number of HTTP requests. With this setting, the queue engine waits up to
<cite>queue.minDequeueBatchSize.timeout</cite> milliseconds when there are fewer messages
currently queued. Note that the minimum batch size cannot be larger than the
configured maximum batch size. If so, it is automatically adjusted to
match the maximum. So if in doubt, you need to specify both parameters.</p>
</div>
<div class="section" id="queue-mindequeuebatchsize-timeout">
<h3>queue.minDequeueBatchSize.timeout<a class="headerlink" href="#queue-mindequeuebatchsize-timeout" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>1000</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<p>This parameter is only meaningful if use together with <cite>queue.minDequeueBatchSize</cite>,
otherwise it is ignored. It specifies the amount of time (in milliseconds) rsyslogs
waits for new
messages so that the minimum batch size can be reached. After this period, the
batch is processed, <em>even if it is below minimum size</em>. This capability exists to
prevent having messages stalled in an incomplete batch just because no new
messages arrive. We would expect that it usually makes little sense to set it
to higher than 60.000 (60 seconds), but this is permitted. Just be warned that
this potentially delays log processing for that long.</p>
</div>
<div class="section" id="queue-maxdiskspace">
<h3>queue.maxDiskSpace<a class="headerlink" href="#queue-maxdiskspace" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>0</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueMaxDiskSpace</span></code></td>
</tr>
</tbody>
</table>
<p>The maximum size that all queue files together will use on disk. Note
that the actual size may be slightly larger than the configured max,
as rsyslog never writes partial queue records.</p>
</div>
<div class="section" id="queue-highwatermark">
<h3>queue.highWatermark<a class="headerlink" href="#queue-highwatermark" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>90% of queue.size</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueHighWaterMark</span></code></td>
</tr>
</tbody>
</table>
<p>This applies to disk-assisted queues, only. When the queue fills up
to this number of messages, the queue begins to spool messages to
disk. Please note that this should not happen as part of usual
processing, because disk queue mode is very considerably slower than
in-memory queue mode. Going to disk should be reserved for cases
where an output action destination is offline for some period.</p>
</div>
<div class="section" id="queue-lowwatermark">
<h3>queue.lowWatermark<a class="headerlink" href="#queue-lowwatermark" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>70% of queue.size</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueLowWaterMark</span></code></td>
</tr>
</tbody>
</table>
<p>This applies to disk-assisted queues, only. When the high watermark is
reached, the queue will write data to disk. It does so until the low
watermark is reached, then the queue reverts back to in-memory mode.</p>
</div>
<div class="section" id="queue-fulldelaymark">
<h3>queue.fullDelaymark<a class="headerlink" href="#queue-fulldelaymark" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>97% of queue.size</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<p>Number of messages when the queue should block delayable messages.
Messages are NO LONGER PROCESSED until the queue has sufficient space
again. If a message is delayable depends on the input. For example,
messages received via imtcp are delayable (because TCP can push back),
but those received via imudp are not (as UDP does not permit a push back).
The intent behind this setting is to leave some space in an almost-full
queue for non-delayable messages, which would be lost if the queue runs
out of space. Please note that if you use a DA queue, setting the
fulldelaymark BELOW the highwatermark makes the queue never activate
disk mode for delayable inputs. So this is probably not what you want.</p>
</div>
<div class="section" id="queue-lightdelaymark">
<h3>queue.lightDelayMark<a class="headerlink" href="#queue-lightdelaymark" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>70% of queue.size</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<p>If this mark is reached the sender will be throttled if possible. The
main idea to do this is leave some space inside the queue for inputs
like UDP which cannot be throttled - and so any data arriving at
“queue full” would be discarded.</p>
<p>If the special value <cite>0</cite> is used, <cite>queue.LightDelayMark</cite> will be set
to the value of <cite>queue.size</cite>. This effectively <strong>disables</strong> light delay
functionality. This is useful if a queue is not used by non-delayable
inputs like UDP. The special value was introduced in rsyslog 8.1904.0
and is <strong>not</strong> available in earlier versions. There, you can achieve the
same result by setting <cite>queue.LightDelayMark</cite> to a very large value.</p>
</div>
<div class="section" id="queue-discardmark">
<h3>queue.discardMark<a class="headerlink" href="#queue-discardmark" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>80% of queue.size</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueDiscardMark</span></code></td>
</tr>
</tbody>
</table>
<p>Specifies the threshold at which rsyslog begins to discard less important
messages. To define which messages should be discarded use the
queue.discardseverity parameter.</p>
</div>
<div class="section" id="queue-discardseverity">
<h3>queue.discardSeverity<a class="headerlink" href="#queue-discardseverity" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>8</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueDiscardSeverity</span></code></td>
</tr>
</tbody>
</table>
<p>As soon as the threshold of the parameter queue.discardMark is reached
incoming as well as queued messages with a priority equal or lower than
specified will be erased. With the default no messages will be erased.
You have to specify a numeric severity value for this parameter.</p>
</div>
<div class="section" id="queue-checkpointinterval">
<h3>queue.checkpointInterval<a class="headerlink" href="#queue-checkpointinterval" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>0</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueCheckpointInterval</span></code></td>
</tr>
</tbody>
</table>
<p>Disk queues by default do not update housekeeping structures every time
the queue writes to disk. This is for performance reasons. In the event of failure,
data will still be lost (except when data is mangled via the file structures).
However, disk queues can be set to write bookkeeping information on checkpoints
(every n records), so that this can be made ultra-reliable, too. If the
checkpoint interval is set to one, no data can be lost, but the queue is
exceptionally slow.</p>
</div>
<div class="section" id="queue-syncqueuefiles">
<h3>queue.syncqueuefiles<a class="headerlink" href="#queue-syncqueuefiles" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>binary</td>
<td>off</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueSyncQueueFiles</span></code></td>
</tr>
</tbody>
</table>
<p>Disk-based queues can be made very reliable by issuing a (f)sync after each
write operation. This happens when you set the parameter to “on”.
Activating this option has a performance penalty, so it should not
be turned on without a good reason. Note that the penalty also depends on
<em>queue.checkpointInterval</em> frequency.</p>
</div>
<div class="section" id="queue-samplinginterval">
<h3>queue.samplingInterval<a class="headerlink" href="#queue-samplinginterval" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>0</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<div class="versionadded">
<p><span class="versionmodified">New in version 8.23.0.</span></p>
</div>
<p>This option allows queues to be populated by events produced at a specific interval.
It provides a way to sample data each N events, instead of processing all, in order
to reduce resources usage (disk, bandwidth…)</p>
</div>
<div class="section" id="queue-type">
<h3>queue.type<a class="headerlink" href="#queue-type" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>word</td>
<td>Direct</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueType</span></code></td>
</tr>
</tbody>
</table>
<p>Specifies the type of queue that will be used. Possible options are “FixedArray”,
“LinkedList”, “Direct” or “Disk”. For more information read the documentation
for <a class="reference internal" href="../concepts/queues.html"><span class="doc">queues</span></a>.</p>
</div>
<div class="section" id="queue-workerthreads">
<h3>queue.workerThreads<a class="headerlink" href="#queue-workerthreads" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>1</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueWorkerThreads</span></code></td>
</tr>
</tbody>
</table>
<p>Specifies the maximum number of worker threads that can be run parallel.</p>
</div>
<div class="section" id="queue-workerthreadminimummessages">
<h3>queue.workerThreadMinimumMessages<a class="headerlink" href="#queue-workerthreadminimummessages" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>queue.size/queue.workerthreads</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueWorkerThreadMinimumMessages</span></code></td>
</tr>
</tbody>
</table>
<p>Specify the number of messages a worker thread is processing before another
worker thread is created. This number is limited by parameter queue.workerThreads.
For example if this parameter is set to 200 and in the queue are 201 messages a
second worker thread will be created.</p>
</div>
<div class="section" id="queue-timeoutworkerthreadshutdown">
<h3>queue.timeoutWorkerthreadShutdown<a class="headerlink" href="#queue-timeoutworkerthreadshutdown" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>60000</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueTimeoutWorkerthreadShutdown</span></code></td>
</tr>
</tbody>
</table>
<p>After starting a worker thread, it will process messages until there are no
messages for him to process. This parameter specifies the time the worker
thread has to be inactive before it times out.
The parameter must be specified in milliseconds. Which means the default of
60000 is 1 minute.</p>
</div>
<div class="section" id="queue-timeoutshutdown">
<h3>queue.timeoutshutdown<a class="headerlink" href="#queue-timeoutshutdown" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>10/1500</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueTimeoutShutdown</span></code></td>
</tr>
</tbody>
</table>
<p>If a queue that still contains messages is terminated it will wait the
specified time interval for the worker thread to finish.
The time is specified in milliseconds (1000ms is 1sec).
Default for action queues is 10, for ruleset queues it is 1500.</p>
</div>
<div class="section" id="queue-timeoutactioncompletion">
<h3>queue.timeoutActionCompletion<a class="headerlink" href="#queue-timeoutactioncompletion" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>1000</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueTimeoutActionCompletion</span></code></td>
</tr>
</tbody>
</table>
<p>When a queue is terminated, the timeout shutdown is over and there is
still data in the queue, the queue will finish the current data element
and then terminate. This parameter specifies the timeout for processing
this last element.
Parameter is specified in milliseconds (1000ms is 1sec).</p>
</div>
<div class="section" id="queue-timeoutenqueue">
<h3>queue.timeoutEnqueue<a class="headerlink" href="#queue-timeoutenqueue" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>2000</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueTimeoutEnqueue</span></code></td>
</tr>
</tbody>
</table>
<p>This timeout value is used when the queue is full. If rsyslog cannot
enqueue a message within the timeout period, the message is discarded.
Note that this is setting of last resort (assuming defaults are used
for the queue settings or proper parameters are set): all delayable
inputs (like imtcp or imfile) have already been pushed back at this
stage. Also, discarding of lower priority messages (if configured) has
already happened. So we run into one of these situations if we do not
timeout quickly enough:</p>
<ul class="simple">
<li>if using imuxsock and no systemd journal is involved, the system
would become unresponsive and most probably a hard reset would be
required.</li>
<li>if using imuxsock with imjournal forwarding is active, messages are
lost because the journal discards them (more aggressive than rsyslog does)</li>
<li>if using imjournal, the journal will buffer messages. If journal
runs out of configured space, messages will be discarded. So in this
mode discarding is moved to a bit later place.</li>
<li>other non-delayable sources like imudp will also loose messages</li>
</ul>
<p>So this setting is provided in order to guard against problematic situations,
which always will result either in message loss or system hang. For
action queues, one may debate if it would be better to overflow rapidly
to the main queue. If so desired, this is easy to accomplish by setting
a very large timeout value. The same, of course, is true for the main
queue, but you have been warned if you do so!</p>
<p>In some other words, you can consider this scenario, using default values.
With all progress blocked (unable to deliver a message):</p>
<ul>
<li><p class="first">all delayable inputs (tcp, relp, imfile, imjournal, etc) will block
indefinantly (assuming queue.lightdelaymark and queue.fulldelaymark
are set sensible, which they are by default).</p>
</li>
<li><p class="first">imudp will be loosing messages because the OS will be dropping them</p>
</li>
<li><p class="first">messages arriving via UDP or imuxsock that do make it to rsyslog,
and that are a severity high enough to not be filtered by
discardseverity, will block for 2 seconds trying to put the message in
the queue (in the hope that something happens to make space in the
queue) and then be dropped to avoid blocking the machine permanently.</p>
<p>Then the next message to be processed will also be tried for 2 seconds, etc.</p>
</li>
<li><p class="first">If this is going into an action queue, the log message will remain
in the main queue during these 2 seconds, and additional logs that
arrive will accumulate behind this in the main queue.</p>
</li>
</ul>
</div>
<div class="section" id="queue-maxfilesize">
<h3>queue.maxFileSize<a class="headerlink" href="#queue-maxfilesize" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>1m/16m</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueMaxFileSize</span></code></td>
</tr>
</tbody>
</table>
<p>Specifies the maximum size for the disk-assisted queue file.
Parameter can be specified in Mebibyte or Gibibyte, default for action
queues is 1m and for ruleset queues 16m (1m = 1024*1024).</p>
</div>
<div class="section" id="queue-saveonshutdown">
<h3>queue.saveOnShutdown<a class="headerlink" href="#queue-saveonshutdown" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>binary</td>
<td>off</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueSaveOnShutdown</span></code></td>
</tr>
</tbody>
</table>
<p>This parameter specifies if data should be saved at shutdown.</p>
</div>
<div class="section" id="queue-dequeueslowdown">
<h3>queue.dequeueSlowDown<a class="headerlink" href="#queue-dequeueslowdown" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>0</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueDequeueSlowDown</span></code></td>
</tr>
</tbody>
</table>
<p>Regulates how long dequeueing should be delayed. This value must be specified
in microseconds (1000000us is 1sec). It can be used to slow down rsyslog so
it won’t send things to fast.
For example if this parameter is set to 10000 on a UDP send action, the action
won’t be able to put out more than 100 messages per second.</p>
</div>
<div class="section" id="queue-dequeuetimebegin">
<h3>queue.dequeueTimeBegin<a class="headerlink" href="#queue-dequeuetimebegin" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>0</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueDequeueTimeBegin</span></code></td>
</tr>
</tbody>
</table>
<p>With this parameter you can specify rsyslog to process queues during specific
time periods. To define a time frame use the 24-hour format without minutes.
This parameter specifies the begin and “queue.dequeuetimeend” the end of the
time frame.</p>
</div>
<div class="section" id="queue-dequeuetimeend">
<h3>queue.dequeueTimeEnd<a class="headerlink" href="#queue-dequeuetimeend" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>integer</td>
<td>25</td>
<td>no</td>
<td><code class="docutils literal"><span class="pre">$ActionQueueDequeueTimeEnd</span></code></td>
</tr>
</tbody>
</table>
<p>With this parameter you can specify rsyslog to process queues during specific
time periods. To define a time frame use the 24-hour format without minutes.
This parameter specifies the end and “queue.dequeuetimebegin” the begin of the
time frame. The default 25 disables the time-window.</p>
</div>
<div class="section" id="queue-takeflowctlfrommsg">
<h3>queue.takeFlowCtlFromMsg<a class="headerlink" href="#queue-takeflowctlfrommsg" title="Permalink to this headline">¶</a></h3>
<table border="1" class="colwidths-auto parameter-table docutils">
<thead valign="bottom">
<tr class="row-odd"><th class="head">type</th>
<th class="head">default</th>
<th class="head">mandatory</th>
<th class="head"><code class="docutils literal"><span class="pre">obsolete</span> <span class="pre">legacy</span></code> directive</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td>boolean</td>
<td>off</td>
<td>no</td>
<td>none</td>
</tr>
</tbody>
</table>
<div class="versionadded">
<p><span class="versionmodified">New in version 8.1911.0.</span></p>
</div>
<p>This is a fine-tuning parameter which permits to control whether or not
rsyslog shall always take the flow control setting from the message. If
so, non-primary queues may also <strong>block</strong> when reaching high water mark.</p>
<p>This permits to add some synchronous processing to rsyslog core engine.
However, <strong>this involves some risk</strong>: Improper use may make the core engine
stall. As such, <strong>enabling this parameter requires very careful planning
of the rsyslog configuration and deep understanding of the consequences</strong>.</p>
<p>Note that the parameter is applied to individual queues, so a configuration
with a large number of queues can (and must if used) be fine-tuned to
the exact use case.</p>
<p><strong>The rsyslog team strongly recommends to let this parameter turned off.</strong></p>
</div>
</div>
<div class="section" id="examples">
<h2>Examples<a class="headerlink" href="#examples" title="Permalink to this headline">¶</a></h2>
<div class="section" id="example-1">
<h3>Example 1<a class="headerlink" href="#example-1" title="Permalink to this headline">¶</a></h3>
<p>The following is a sample of a TCP forwarding action with its own queue.</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>action(type="omfwd" target="192.168.2.11" port="10514" protocol="tcp"
queue.filename="forwarding" queue.size="1000000" queue.type="LinkedList"
)
</pre></div>
</div>
<div class="admonition seealso">
<p class="first admonition-title">See also</p>
<p>Help with configuring/using <code class="docutils literal"><span class="pre">Rsyslog</span></code>:</p>
<ul class="last simple">
<li><a class="reference external" href="http://lists.adiscon.net/mailman/listinfo/rsyslog">Mailing list</a> - best route for general questions</li>
<li>GitHub: <a class="reference external" href="https://github.com/rsyslog/rsyslog/">rsyslog source project</a> - detailed questions, reporting issues
that are believed to be bugs with <code class="docutils literal"><span class="pre">Rsyslog</span></code></li>
<li>Stack Exchange (<a class="reference external" href="https://stackexchange.com/filters/327462/rsyslog">View</a>, <a class="reference external" href="https://serverfault.com/questions/ask?tags=rsyslog">Ask</a>)
- experimental support from rsyslog community</li>
</ul>
</div>
<div class="admonition seealso">
<p class="first admonition-title">See also</p>
<p>Contributing to <code class="docutils literal"><span class="pre">Rsyslog</span></code>:</p>
<ul class="last simple">
<li>Source project: <a class="reference external" href="https://github.com/rsyslog/rsyslog/blob/master/README.md">rsyslog project README</a>.</li>
<li>Documentation: <a class="reference external" href="https://github.com/rsyslog/rsyslog-doc/blob/master/README.md">rsyslog-doc project README</a></li>
</ul>
</div>
<p>Copyright 2008-2020 <a class="reference external" href="https://rainer.gerhards.net/">Rainer Gerhards</a>
(<a class="reference external" href="https://www.rainer-gerhards.de/grossrinderfeld/">Großrinderfeld</a>),
and Others.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">General Queue Parameters</a><ul>
<li><a class="reference internal" href="#usage">Usage</a></li>
<li><a class="reference internal" href="#configuration-parameters">Configuration Parameters</a><ul>
<li><a class="reference internal" href="#queue-filename">queue.filename</a></li>
<li><a class="reference internal" href="#queue-spooldirectory">queue.spoolDirectory</a></li>
<li><a class="reference internal" href="#queue-size">queue.size</a></li>
<li><a class="reference internal" href="#queue-dequeuebatchsize">queue.dequeueBatchSize</a></li>
<li><a class="reference internal" href="#queue-mindequeuebatchsize">queue.minDequeueBatchSize</a></li>
<li><a class="reference internal" href="#queue-mindequeuebatchsize-timeout">queue.minDequeueBatchSize.timeout</a></li>
<li><a class="reference internal" href="#queue-maxdiskspace">queue.maxDiskSpace</a></li>
<li><a class="reference internal" href="#queue-highwatermark">queue.highWatermark</a></li>
<li><a class="reference internal" href="#queue-lowwatermark">queue.lowWatermark</a></li>
<li><a class="reference internal" href="#queue-fulldelaymark">queue.fullDelaymark</a></li>
<li><a class="reference internal" href="#queue-lightdelaymark">queue.lightDelayMark</a></li>
<li><a class="reference internal" href="#queue-discardmark">queue.discardMark</a></li>
<li><a class="reference internal" href="#queue-discardseverity">queue.discardSeverity</a></li>
<li><a class="reference internal" href="#queue-checkpointinterval">queue.checkpointInterval</a></li>
<li><a class="reference internal" href="#queue-syncqueuefiles">queue.syncqueuefiles</a></li>
<li><a class="reference internal" href="#queue-samplinginterval">queue.samplingInterval</a></li>
<li><a class="reference internal" href="#queue-type">queue.type</a></li>
<li><a class="reference internal" href="#queue-workerthreads">queue.workerThreads</a></li>
<li><a class="reference internal" href="#queue-workerthreadminimummessages">queue.workerThreadMinimumMessages</a></li>
<li><a class="reference internal" href="#queue-timeoutworkerthreadshutdown">queue.timeoutWorkerthreadShutdown</a></li>
<li><a class="reference internal" href="#queue-timeoutshutdown">queue.timeoutshutdown</a></li>
<li><a class="reference internal" href="#queue-timeoutactioncompletion">queue.timeoutActionCompletion</a></li>
<li><a class="reference internal" href="#queue-timeoutenqueue">queue.timeoutEnqueue</a></li>
<li><a class="reference internal" href="#queue-maxfilesize">queue.maxFileSize</a></li>
<li><a class="reference internal" href="#queue-saveonshutdown">queue.saveOnShutdown</a></li>
<li><a class="reference internal" href="#queue-dequeueslowdown">queue.dequeueSlowDown</a></li>
<li><a class="reference internal" href="#queue-dequeuetimebegin">queue.dequeueTimeBegin</a></li>
<li><a class="reference internal" href="#queue-dequeuetimeend">queue.dequeueTimeEnd</a></li>
<li><a class="reference internal" href="#queue-takeflowctlfrommsg">queue.takeFlowCtlFromMsg</a></li>
</ul>
</li>
<li><a class="reference internal" href="#examples">Examples</a><ul>
<li><a class="reference internal" href="#example-1">Example 1</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="lookup_tables.html"
title="previous chapter">Lookup Tables</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="rainerscript_call.html"
title="next chapter">The rsyslog “call” statement</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/rainerscript/queue_parameters.rst.txt"
rel="nofollow">Show Source</a></li>
<li><a href="https://github.com/rsyslog/rsyslog-doc/edit/master/source/rainerscript/queue_parameters.rst"
rel="nofollow">Edit on GitHub</a></li>
</ul>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="rainerscript_call.html" title="The rsyslog “call” statement"
>next</a> |</li>
<li class="right" >
<a href="lookup_tables.html" title="Lookup Tables"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">rsyslog 8.2006.0 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="../configuration/index.html" >Configuration</a> »</li>
<li class="nav-item nav-item-2"><a href="index.html" >RainerScript</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
</div>
</body>
</html>