\chapter{Random number generation}
\label{chap:randgen}

The ability to generate pseudo-random numbers (random number
generation, or RNG) is indispensable for modern statistical
methods. In this chapter we provide some details on the functions
gretl offers in this area, with a brief description of the most
important algorithms it uses.

\section{Uniform and normal pseudo-random variates}
\label{sec:randgen-uni-norm}

\subsection{The uniform distribution}
\label{sec:randgen-uni}

The cornerstone of pseudo-random number generation is the ability to
generate high-quality uniform variates, that is, random numbers $U$ in
the unit interval such that $P(a \le U < b) = b - a$ for
$0 \le a \le b \le 1$. As is well known, pseudo-random numbers generated by
software such as gretl are not truly random. They are practically
indistinguishable from genuine random numbers, but are in fact
generated sequentially via a deterministic algorithm.

This means that one can set the starting point of the sequence and
obtain a repeatable sequence of random numbers. Each algorithm has a
\emph{cycle length}, or \emph{period}, the number of draws after which
the sequence starts repeating itself. Clearly, the cycle length should
exceed the number of pseudo-random numbers that one might plausibly
wish to generate.  The RNG that gretl now uses for generating uniforms
is \textsf{xoshiro256+} by \citet{blackman-vigna22}; its period is
$2^{256} - 1 \approx 1.158 \times 10^{77}$. See
\url{https://prng.di.unimi.it/} for further details.\footnote{In
  releases up to gretl 2025c we used the SIMD-oriented Fast Mersenne
  Twister \citep{saito_matsumoto08}.}

The starting point of the sequence is also called the \emph{seed} of
the generator, and it can be set by using the \cmd{set seed} command.
The seed must be expressible as a 64-bit unsigned integer, lying
between 0 and $2^{64} - 1 \approx 1.845 \times 10^{19}$. If the user
does not provide a seed, gretl sets it automatically, using entropy
from \texttt{/dev/urandom} if available otherwise the so-called ``Unix
time'', that is the number of seconds elapsed between 1970-01-01
00:00:00 UTC and the time at which gretl is started.

The `raw' output of \textsf{xoshiro256+} consists of pseudo-random
unsigned integers from the uniform distribution on the interval
$[0, 2^{64}-1]$. These are unlikely to be wanted in econometric work;
since all numeric data in gretl is stored in double precision format,
the RNG is used for producing real numbers between 0 and 1 on an
equispaced grid of $2^{53}$ values, which roughly corresponds to 16
decimal digits.

Gretl offers two functions for producing uniform variates on
$[0,1)$: \cmd{uniform}, which produces a series, and \cmd{muniform},
which produces a matrix, (see the \GCR\ for the syntax details).  Of
course, the former function can be used only if a dataset is in place.

For example, the code
\begin{code}
  set seed 1234567890
  printf "%16.12f", muniform(3,2)
\end{code}
in which the seed is set by the user, should invariably produce the
matrix
\begin{code}
  0.088025327009  0.547567021443
  0.428678844208  0.307482010724
  0.600651539849  0.955673248007
\end{code}
If you wish to reset the seed to something over which you have no
control, you can use the command \cmd{set seed auto}.

\subsection{The normal distribution}
\label{sec:randgen-norm}

The second most important distribution in econometric computation is
the normal distribution: gretl offers the \cmd{normal} function for
series and the \cmd{mnormal} function for matrices, which produce
standard Gaussian variates. The algorithm we use is the ``Ziggurat''
method originally devised by \cite{marsaglia00}; gretl has adopted the
refinements introduced by David Bateman in the \textsf{Octave}
implementation. The normal equivalent to the code above is
\begin{code}
  set seed 1234567890
  printf "%16.12f", mnormal(3,2)
\end{code}
whose output is
\begin{code}
 -0.300383700185  0.265822214364
  0.816693512224  0.836117005449
  0.147805465748  0.644932184014
\end{code}

Drawing from a generic normal distribution with arbitrary mean and
covariance matrix is easy by using the linearity property of the
normal distribution. If you need pseudo-random numbers drawn from a
normal density with mean $\mu$ and covariance matrix $\Sigma$, you can
perform an affine transformation, as in
\[
  X \sim N(0, I) \Longrightarrow AX + b  \sim N(b, AA')
\]
Therefore to draw from a $N(\mu, \Sigma)$ distribution you need a
matrix whose product by its own transpose equals $\Sigma$, the
Cholesky decomposition being a popular choice. For example,
\begin{code}
  set seed 1234567890
  scalar N = 8192
  matrix X = mnormal(N,2)
  matrix m = {3,4}
  matrix S = {4,2;2,2}
  matrix K = cholesky(S)
  matrix Z = X * K' .+ m
  eval meanc(Z)
  eval mcov(Z)
\end{code}
will produce
\begin{code}
      2.9666       3.9788 

      4.0689       2.0165 
      2.0165       1.9856 
\end{code}
Note that in this example the individual draws are the \emph{rows} of
the matrices $X$ and $Z$, so we are using a row vector $m$ and the
transpose of the Cholesky factor $K$.\footnote{The \cmd{.+} operator
  is used to add the \texttt{m} row vector to each row of the
  \texttt{X * K'} matrix.  For more details on matrix operations, see
  Chapter \ref{chap:matrices} in the \GUG.}

When you use the \cmd{normal} function for producing a series, the
above transformation can be performed via two optional arguments,
denoting mean and standard deviation, respectively. The following code
demonstrates the equivalence of the two methods:
\begin{code}
set seed 1234567890
nulldata 8192
scalar m = 3
scalar s = 2
series x1 = normal() * s + m
series x2 = normal(m, s)
summary x1 x2 --simple
\end{code}
The output of the above is
\begin{code}
           Mean      Median        S.D.         Min         Max
x1       2.9666      2.9816      2.0171     -4.7079      9.6585
x2       2.9911      3.0221      1.9862     -5.2539      9.8079
\end{code}

\section{Other distributions via native functions}
\label{sec:randgen-other-native}

Gretl offers several built-in algorithms for generating pseudo-rvs
efficiently from a reasonably large array of distributions. These
algorithms are accessed via three different functions, which share a
similar syntax, that is
\begin{flushleft}
  \emph{\texttt{function(distributional arguments[, dimensional arguments])}}
\end{flushleft}
where \emph{\texttt{function}} can be
\begin{itemize}
\item \cmd{randgen1} for generating scalars,
\item \cmd{randgen} for generating series and
\item \cmd{mrandgen} for generating matrices.
\end{itemize}

The distributional arguments have to be passed first, to identify the
distribution from which you want to draw numbers. The first one must
be a string identifying the distribution; extra numerical parameters
may be needed to pin down its exact shape (e.g.\ one
degrees-of-freedom term for the $\chi^2$ density and two for the $F$
density). For example, suppose you want to generate a single draw from
a $t$ distribution with 12 degrees of freedom; the command would be
something like
\begin{code}
  scalar foo = randgen1(t, 12)
\end{code}
where \texttt{t} identifies the Student's $t$ density. The
\cmd{mrandgen} function needs two additional parameters (the
\emph{\texttt{dimensional arguments}} noted above) for the size of the
returned matrix. The following example generates a $4 \times 3$ matrix
of pseudo-random numbers from a Poisson distribution with parameter 2:
\begin{code}
  matrix bar = mrandgen(P, 2, 4, 3)
\end{code}

Tables \ref{tab:discrete} and \ref{tab:continuous} contain a summary
of the supported distributions and their arguments.\footnote{In both
  tables, $B(a,b) = \frac{\Gamma(a) \Gamma(b)}{\Gamma(a+b)}$ is 
    Euler's Beta function.}

\begin{table}[htbp]
\centering
\begin{tabular}{rccl}
  \hline
  distribution	& code	& $P(x) = k$	& extra parameters \\
  \hline
Discrete uniform	& \texttt{i}	& $\frac{1}{M - m +1}$	& $m, M$ \\[6pt]
Binomial		& \texttt{b}	& ${n \choose k} p^k (1-p)^{(n-k)}$	& $p, n$ \\[6pt]
Poisson			& \texttt{p}	& $e^{-\mu} \frac{\mu^k}{k!}$	& $\mu$ \\[6pt]
Beta-Binomial		& \texttt{bb}	& ${n \choose k} \frac{B(k+\alpha,n-k+\beta)}
                          {B(\alpha,\beta)}$ & $n, \alpha$, $\beta$ \\[6pt]
Generic discrete	& \texttt{disc}	& $p_k$	& $\mathbf{p}$ \\[6pt]
  \hline
  \end{tabular}\
\caption{Discrete distributions}
\label{tab:discrete}
\end{table}

\begin{table}[htbp]
\centering
\begin{tabular}{rccl}
  \hline
  distribution	& code	& $f(x)$	& extra parameters \\
  \hline
  Uniform	& \texttt{u}
      & $\frac{1}{b-a}$	&  $a,b$\\[9pt]
  Normal	& \texttt{z}
      & $\frac{1}{\sigma \sqrt{2\pi}}
      \exp\left[-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2\right]
        = \frac{1}{\sigma} \varphi\left[ \frac{x-\mu}{\sigma} \right]$
      & $\mu, \sigma$ \\[9pt]
  Truncated normal & \texttt{tn}
      & $\frac{\varphi(x)}{\Phi(h) - \Phi(l)}$
      & $l, h$ \\[9pt]
  Student's $t$	& \texttt{t}
      & $\frac{\Gamma{\left(\frac{\nu + 1}{2}\right)}}{\sqrt{\pi\nu}\,
        \Gamma{\left(\frac{\nu}{2}\right)}}
        \left(1 + \frac{x^2}{\nu}\right)^{-\frac{\nu+1}{2}}$
      & $\nu$ \\[9pt]
  $\chi^2$	& \texttt{x}
      & $\frac{1}{2^{p/2}\Gamma(p/2)}\; x^{(p/2)-1} e^{-x/2}$
      &  $p$ \\[9pt]
  Snedecor's $F$	& \texttt{f}
      & $\dfrac{(d_1/d_2)^{d_1/2} x^{(d_1/2)-1}}
       {B(d_1/2, d_2/2)[1+(d_1x/d_2)]^{(d_1+d_2)/2}}$
      & $d_1, d_2$ \\[9pt]
  Exponential	& \texttt{exp}
      & $\frac{e^{-x/\mu}}{\mu}$	& $\mu$ \\[9pt]
  Weibull	& \texttt{w}
      & $\frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1}
        e^{-(x/\lambda)^{k}}$
      & $k, \lambda$ \\[9pt]
  Gamma	& \texttt{g}
      & $\frac{ \alpha^{-p}}{\Gamma(p)} x^{p - 1} e^{-x/\alpha}$

      & $p, \alpha$ \\[9pt]
  Generalized Error	& \texttt{e}
      & $\frac{\nu}{2^{1+1/\nu} \Gamma(1/\nu)} \exp[{-\frac{1}{2}|x|^\nu}]$
      & $\nu$  \\[9pt]
  Laplace	& \texttt{l}
     & $\frac{1}{2 h} \exp[{-\frac{|x - \mu|}{h}}]$
     & $\mu, h$ \\[9pt]
  Beta	& \texttt{beta}
     & $\frac{1}{B(\alpha,\beta)} x^{\alpha-1}(1-x)^{\beta-1}$
     & $\alpha, \beta$	\\[9pt]
  Logistic	& \texttt{s}
     & $\frac{e^{-(x-\mu)/s}} {s\left(1+e^{-(x-\mu)/s}\right)^2}$

     & $\mu, s$ \\
  \hline
  \end{tabular}
  \medskip\par
  \caption{Univariate continuous distributions}
\label{tab:continuous}
\end{table}

In addition to the univariate distributions listed in Table
\ref{tab:continuous}, two multivariate distributions are supported via
special syntax: 
\begin{itemize}
\item The Dirichlet distribution is available via \cmd{mrandgen}, with
  a variation in syntax reflecting its multivariate nature. The first
  argument is \texttt{dir} and the second must be a vector of
  $p \ge 2$ distributional parameters (positive reals); the third is
  an integer, $n$, specifying the number of draws. It returns a
  $n \times p$ matrix, each row of which is an independent draw.
\item For the Inverse-Wishart distribution, which is defined on
  real-valued positive-definite matrices, we have the dedicated
  function \cmd{iwishart}. This takes two arguments, a $p\times p$
  positive definite scale matrix and an integer degrees-of-freedom
  term. The returned matrix, computed using the method of
  \cite{odell-feiveson66}, is also $p\times p$. This distribution is
  often used in Bayesian methods as the conjugate prior for the
  covariance matrix of a multivariate normal distribution.
\end{itemize}

\section{Other distributions via scripting}
\label{sec:randgen-other-hansl}

Other distributions can often be handled by methods that are fairly
straightforward to code in hansl: a simple and general one is the
``inversion method'': if $X$ is a random variable with distribution
function $F$, then $U = F(X)$ is a uniform random variable. Therefore,
one method for generating variates from a distribution with function
$F$---assuming that the inverse function $F^{-1}$ is readily
computable---is to generate uniform variates and apply the inverse
transformation:
\[
  X = F^{-1}(U) .
\]
For example, take a Beta distribution with parameter $\alpha = 2$ and
$\beta=1$, that is
\[
  f(x) = 2 x
\]
Its distribution function can be written as
\[
  F(z) = P(x \le z) = \int_{0}^z 2 x\ \mathrm{d}x = \left[ x^2\right]_0^z = z^2;
\]
therefore, by inverting this function, it follows that $X$ must be
such that
\[
  U = X^2,
\]
where $U$ is a uniform random variable, so
\[
  X = \sqrt{U}
\]
will follow the desired distribution, and the hansl command to
generate $X$ is simply
\begin{code}
  series x = sqrt(uniform())
\end{code}

% For example, take the geometric distribution with parameter $\alpha$,
% that is
% \[
%   p(x) = (1-\alpha) \cdot \alpha^x
% \]
% Its distribution function can be written as
% \[
%   F(z) = P(x \le z) = (1-\alpha) \sum_{x=0}^z \alpha^x = (1-\alpha)
%   \frac{1-\alpha^{z+1}}{1-\alpha} = 1-\alpha^{z+1} ;
% \]
% therefore, by inverting this function, it follows that $X$ must be
% such that
% \[
%   1 - \alpha^{X+1} = U^*,
% \]
% where $U^*$ is a uniform random variable; but then, if $U^* \sim
% U(0,1)$, the same goes for $U = 1 - U^*$. Therefore,
% \[
%   \alpha^{X+1} = U \Longrightarrow (X+1) \log \alpha = \log U
% \]
% \[
%   X = \left\lfloor \frac{\log{U}}{\log{\alpha}} \right\rfloor
% \]
% will follow a geometric distribution. A corresponding script is
% \begin{code}
%   nulldata 1024
%   alpha = 0.75
%   series x = floor(log(uniform())/log(alpha))
% \end{code}

More complex methods, such as acceptance--rejection or
ratio-of-uniforms, can also be easily implemented as hansl
functions. For further details, see \cite{Gentle2004},
\cite{Monahan2001}, chapter 11 or \cite{Tanizaki2004}, chapters 2--3.

\section{Usage with MPI}
\label{sec:mpi}

A special issue arises when a task that involves generating random
numbers is parallelized across several processes, as in \textsf{MPI}
(Message Passing Interface). Details on gretl's support for
\textsf{MPI}, and how to use it, can be found in \cite{cottrell-mpi};
here we'll just say a little about the RNG aspect.

By default we assume that the random sequences generated by multiple MPI
processes should be mutually independent. With \textsf{xoshiro256+} that
is easily accomplished: at the start-up of MPI, process 0 obtains a seed
and broadcasts it to the other processes, each of which calls
\textsf{xoshiro}'s \texttt{jump()} function, which advances the RNG by
$2^{128}$ steps. Process 1 calls \texttt{jump} once, process 2 calls it
twice, and so on. This means that the RNG for each process has an
effective period of $2^{128} \approx 3.403 \times 10^{38}$ without any
overlap with other processes.

If for some reason you want each MPI process to generate the
\textit{same} sequence of pseudo-random variates you can use the
\option{single-rng} option when invoking \textsf{gretlmpi}, or at the
end of an \texttt{mpi} block in a gretl script. To switch to a single
RNG in the course of an MPI-enabled script it's sufficient to put a
\texttt{set seed} command with a specific seed in a common portion of
the script, where it will be executed by all processes.

\section{The Gibbs sampler}
\label{sec:gibbs}

It's probably no exaggeration to say that the surge in popularity of
Bayesian methods from the 1990s onwards in largely due to the
possibility of generating pseudo-random numbers from a very wide range
of (possibly very intricate) distributions. This was enabled by cheap,
fast hardware but, more importantly, by efficient sampling
algorithms, one of the most celebrated of which is the \emph{Gibbs
  sampler}---see \cite{gelfand90,casella92}.

Roughly speaking, the Gibbs sampler is used for sampling in a rather
common situation, where you have a multivariate distribution in which
the marginal distributions may be complicated to write, but the
\emph{conditional} distribution of each of its elements is a simple
one, \emph{conditional on the other ones}.

The algorithm is iterative, making it a special case of Markov chain
Monte Carlo (MCMC). For example, suppose we want to generate values
for a triple of random variables $(X, Y, Z)$. The algorithm proceeds
as follows:
\begin{enumerate}
\item set $X_0, Y_0$ and $Z_0$ to something in their respective support.
\item Draw $X_i$ from $f(X|Y_{i-1}, Z_{i-1})$
\item Draw $Y_i$ from $f(Y|X_i, Z_{i-1})$
\item Draw $Z_i$ from $f(Z|X_i, Y_i)$
\item repeat from 2 many times
\item discard the first draws (the so-called \emph{burn-in} stage)
\end{enumerate}

What you get in the end is draws from the unconditional densities
of $X$, $Y$ and $Z$. For example, suppose one wants to draw from a
bivariate normal distibution with the following parameters:
\[
  \left( \begin{array}{c} Y \\ X \end{array} \right) \sim
  N \left[
  \left( \begin{array}{c} 3 \\ 1 \end{array} \right), 
  \left( \begin{array}{cc} 2 & 1 \\ 1 & 4 \end{array} \right)
  \right]
\]
Of course, it is perfectly possible (and numerically preferable) to
use the direct method described in Section \ref{sec:randgen-uni-norm},
but if one wanted to use the Gibbs sampler, the starting point would
be writing down the conditional distributions:
\begin{eqnarray*}
  Y|X & \sim & N\left( 3 + \frac{X-1}{4},\  2 - \frac{1}{4} \right) \\
  X|Y & \sim & N\left( 1 + \frac{Y-3}{2},\  4 - \frac{1}{2} \right)
\end{eqnarray*}

The tool gretl offers for this purpose is the ``gibbs block'', that is
a block of statements which begins with \cmd{gibbs} and ends with
\cmd{end gibbs}.

On the first line, the number of draws and the burn-in period must be
indicated via the \cmd{N} and \cmd{burnin} keywords (either as
numerical values or via the names of pre-existing scalar variables). The 
the \cmd{output} argument indicates the name of the bundle that will
eventually contain the results.

The interior lines of the block can have a keyword prepended, either
\texttt{init} or \texttt{record}, with the following meanings.
\begin{itemize}
\item \cmd{init} identifies a statement as an initializer, to be
  executed just once prior to the Gibbs iteration.
\item \cmd{record} tags a statement as one whose result should be
  recorded at each iteration.
\end{itemize}

Statements that feature neither keyword are taken to be part of the
iteration but their results are not recorded in the output
bundle. Statements tagged with \texttt{init} should be given first.

The primary content of the output bundle is a matrix named \texttt{H}
which by default has $N$ rows and a column dimension that depends on
the number and type of the marginals to be recorded; matrix results
are recorded in transposed vec form. Besides this matrix the bundle
contains several items of metadata which make it a self-contained
record of the sampler. If the bundle named in the output argument
already exists it is overwritten, otherwise a new bundle is created.

The code for generating a double normal follows:

\begin{code}
nulldata 10000
set seed 1234567890

ndraws = $nobs
gibbs burnin=100 N=ndraws output=b
    init x = 0
    m = (x-1)/4 + 3
    s = sqrt(2 - 1/4)
    record y = randgen1(z, m, s) 
    m = (y-3)/2 + 1
    s = sqrt(4 - 1/2)
    record x = randgen1(z, m, s)
end gibbs

series Y = b.H[,1]
series X = b.H[,2]
\end{code}
%$
\begin{figure}[htbp]
  \centering
  \includegraphics[scale=0.5]{figures/dnormal-density.png}
  \caption{Estimated density of the generated data}
  \label{fig:dnormal-density}
\end{figure}

Here's the output (Figure \ref{fig:dnormal-density} displays a kernel
estimate of the joint density of the generated data).

\begin{code}
scalar y
             Mean      Median        S.D.         Min         Max
col1       2.9916      2.9959      1.4026     -3.3100      7.9858

scalar x
             Mean      Median        S.D.         Min         Max
col2      0.97753     0.98632      2.0132     -5.9734      8.7937
\end{code}

The sequential nature of the algorithm does not guarantee that the
draws will be serially independent: in the example above, $Y_i$ and
$Y_{i+1}$ are drawn from the same marginal distribution, but they need
not necessarily be independent, although the Markov nature of the
Gibbs sampler guarantees that the degree of mutual dependence between
$Y_i$ and $Y_{i+k}$ should decrease with $k$. For example, Figure
\ref{fig:Y-ACF} displays the autocorrelation function of the values
for $Y$ generated by the previous example.

\begin{figure}[htbp]
  \centering
  \includegraphics[scale=0.75]{figures/Y-ACF}
  \caption{Autocorrelation function of the generated $Y$}
  \label{fig:Y-ACF}
\end{figure}

In order to overcome this problem one can specify that only every
$k^{\rm th}$ draw obtained via the algorithm above is retained, the
others being skipped. This is handled by the \option{thinning} option,
which goes on the same line as the \texttt{end gibbs} command and
specifies the value of $k$. For example, a command block like the
following
\begin{code}
gibbs burnin=100 N=1000 output=outb
...
end gibbs --thinning=5
\end{code}
produces 1000 post-burnin draws, but only keeps every fifth one, so the
matrix \cmd{H} contained in the output bundle \texttt{outb} contains
200 rows.

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "gretl-guide"
%%% End:
