@c This file is generated automatically by the packages munge-texi.pl.

\input texinfo
@c %**start of header
@setfilename parallel.info
@settitle parallel_doc
@c %**end of header

@c Nowadays the predined function index has entries for each @deftypefn
@c in additiont to each @findex.
@defcodeindex mfn

@copying
General documentation for the parallel package for Octave.

Copyright @copyright{} @email{Olaf Till <i7tiol@@t-online.de>}

You can redistribute this documentation and/or modify it under the terms
of the GNU General Public License as published by the Free Software
Foundation; either version 3 of the License, or (at your option) any
later version.

This documentation is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
Public License for more details.

You should have received a copy of the GNU General Public License along
with this documentation; if not, see <http://www.gnu.org/licenses/>.
@end copying

@include macros.texi

@macro mysee
@ifhtml
see
@end ifhtml
@end macro

@titlepage
@title General documentation for the documentation package for Octave
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@c No table of contents. The table would occupy most of the top node in
@c html and IMHO misleads the user to use the table instead of the menu
@c structure of the nodes, which would let some information unused.
@c
@c @contents

@c ------------------------------------------------------------------

@node Top
@top General documentation for the parallel package for Octave

@ifhtml
The info version of this document is accessible, after package
installation, from the Octave commandline with @code{parallel_doc()}.
@end ifhtml

This documentation applies to version 3.1.3 of the parallel
package.

The package contains functions for explicit local parallel execution,
and functions for parallel execution over a cluster of machines,
possibly combined with local parallel execution at each machine of the
cluster.

The cluster-related functions may be disabled in distributed binaries of
the parallel package for some operating systems.

An alternative to the parallel package is the mpi package.

@menu
* Installation::                Installation hints.
* Local execution::             Functions for local parallel execution.
* Cluster execution::           Functions for parallel execution over a
                                  cluster of machines.
* Further functions::           Functions possibly helpful in parallel
                                  execution.
* Documentation::               Function parallel_doc to view
                                  documentation.

Indices
* Function index::              Index of functions in database.
* Concept index::               Concept index.
@end menu

@c ------------------------------------------------------------------

@node Installation
@chapter Installation hints
@cindex installation

@emph{Note:} For using a cluster of machines, identical versions of the
package are required to be installed at each machine.

Maybe your distribution provides the parallel package.

If not, or if you want a newer version, Octaves @code{pkg} command
allows building the package from source and installing it. Note that you
have to @code{load} any Octave package before you can use it. See
Octaves documentation of @code{pkg}.

The functions for parallel execution over a cluster of machines can be
disabled by a configure option. There is currently no way, however, to
specify configure options when building a package with @code{pkg}.

Parallel execution over a cluster by default is protected by encryption
with authentication. This needs a version of the gnutls library in which
TLS-SRP is not disabled. For building the parallel package from source,
this library must be present during the build; some operating systems
provide development files of libraries in exta @code{...-dev} packages,
which are also needed during the build.

Building or using the package for cluster execution without gnutls will
work, but encryption and authentication are not available in this case.

@c ------------------------------------------------------------------

@node Local execution
@chapter Functions for local parallel execution
@cindex local execution

Explicit local parallel execution, with the intent to exploit more than
one local processor(-core), is performed by calling a user-defined
function in parallel with several different arguments. This is done in
parallel @emph{processes}, so any changes to (global) variables in the
user-defined function will only be visible within the same function
call.

The interface of the functions for local parallel execution is similar
to Octaves @code{cellfun} and @code{parcellfun} functions.

Note that some operations in Octave, particularly some matrix
operations, may already be performed in parallel threads. This may limit
the advantage yielded by explicit local parallel execution. Also, RAM
access can be a bottleneck which limits computation speed of multicore
computing.

@menu
* parcellfun::                        Function parcellfun.
* pararrayfun::                       Function pararrayfun.
@end menu

@c ------------------------------------------------------------------

@node parcellfun
@section Function parcellfun
@mfnindex parcellfun

@c include function helptext here
@c parcellfun ../inst/parcellfun.m
@anchor{XREFparcellfun}
@deftypefn{Function File} {[@var{o1}, @var{o2}, @dots{}] =} parcellfun (@var{nproc}, @var{fun}, @var{a1}, @var{a2}, @dots{})
@deftypefnx{Function File} {} parcellfun (nproc, fun, @dots{}, "UniformOutput", @var{val})
@deftypefnx{Function File} {} parcellfun (nproc, fun, @dots{}, "ErrorHandler", @var{errfunc})
@deftypefnx{Function File} {} parcellfun (nproc, fun, @dots{}, "VerboseLevel", @var{val})
@deftypefnx{Function File} {} parcellfun (nproc, fun, @dots{}, "ChunksPerProc", @var{val})
@deftypefnx{Function File} {} parcellfun (nproc, fun, @dots{}, "CumFunc", @var{cumfunc})
Evaluates a function for multiple argument sets using multiple processes.
@var{nproc} should specify the number of processes. A maximum recommended value is
equal to number of CPUs on your machine or one less.
@var{fun} is a function handle pointing to the requested evaluating function.
@var{a1}, @var{a2} etc. should be cell arrays of equal size.
@var{o1}, @var{o2} etc. will be set to corresponding output arguments.

The UniformOutput and ErrorHandler options are supported with meaning identical
to @dfn{cellfun}.
A VerboseLevel option controlling the level output is supported.
A value of 0 is quiet, 1 is normal, and 2 or more enables
debugging output.
The ChunksPerProc option control the number of chunks which contains elementary jobs. This
option particularly useful when time execution of function is small. Setting this option
to 100 is a good choice in most cases.

Instead of returning a result for each argument, parcellfun returns
only one cumulative result if "CumFunc" is non-empty. @var{cumfunc}
must be a function which performs an operation on two sets of
outputs of @var{fun} and returnes as many outputs as @var{fun}. If
@var{nout} is the number of outputs of @var{fun}, @var{cumfunc}
gets a previous output set of @var{fun} or of @var{cumfunc} as
first @var{nout} arguments and the current output of @var{fun} as
last @var{nout} arguments. The performed operation must be
mathematically commutative and associative. If the operation is
e.g. addition, the result will be the sum(s) of the outputs of
@var{fun} over all calls of @var{fun}. Since floating point
addition and multiplication are only approximately associative, the
cumulative result will not be exactly reproducible.

Notice that jobs are served from a single first-come first-served queue,
so the number of jobs executed by each process is generally unpredictable.
This means, for example, that when using this function to perform Monte-Carlo
simulations one cannot expect results to be exactly reproducible.  The pseudo
random number generators of each process are initialised with a unique state.
This currently works only for new style generators.

NOTE: this function is implemented using "fork" and a number of pipes for IPC.
Suitable for systems with an efficient "fork" implementation (such as GNU/Linux),
on other systems (Windows) it should be used with caution.
Also, if you use a multithreaded BLAS, it may be wise to turn off multi-threading
when using this function.

CAUTION: This function should be regarded as experimental. Although all subprocesses
should be cleared in theory, there is always a danger of a subprocess hanging up,
especially if unhandled errors occur. Under GNU and compatible systems, the following
shell command may be used to display orphaned Octave processes:
ps --ppid 1 | grep octave

@end deftypefn


@c ------------------------------------------------------------------

@node pararrayfun
@section Function pararrayfun
@mfnindex pararrayfun

@c include function helptext here
@c pararrayfun ../inst/pararrayfun.m
@anchor{XREFpararrayfun}
@deftypefn{Function File} {[@var{o1}, @var{o2}, @dots{}] =} pararrayfun (@var{nproc}, @var{fun}, @var{a1}, @var{a2}, @dots{})
@deftypefnx{Function File} {} pararrayfun (nproc, fun, @dots{}, "UniformOutput", @var{val})
@deftypefnx{Function File} {} pararrayfun (nproc, fun, @dots{}, "ErrorHandler", @var{errfunc})
Evaluates a function for corresponding elements of an array. 
Argument and options handling is analogical to @code{parcellfun}, except that
arguments are arrays rather than cells. If cells occur as arguments, they are treated
as arrays of singleton cells.
Arrayfun supports one extra option compared to parcellfun: "Vectorized".
This option must be given together with "ChunksPerProc" and it indicates
that @var{fun} is able to operate on vectors rather than just scalars, and returns
a vector. The same must be true for @var{errfunc}, if given.
In this case, the array is split into chunks which are then directly served to @var{func}
for evaluation, and the results are concatenated to output arrays.
If "CumFunc" is also specified (see @code{parcellfun}), @var{fun} is
expected to return the result of the same cumulative operation
instead of vectors.
@c Will be cut out in parallels info file and replaced with the same
@c references explicitely there, since references to core Octave
@c functions are not automatically transformed from here to there.
@end deftypefn


See also @ref{XREFparcellfun,,parcellfun},
@ref{XREFarrayfun,,arrayfun,octave}.

@c ------------------------------------------------------------------

@node Cluster execution
@chapter Functions for parallel execution over a cluster of machines
@cindex cluster execution

There are high level functions, similar to @ref{parcellfun} and
@ref{pararrayfun}, which only require the user to provide a function to
be called and a series of argument sets. Parallel function calls are
then internally distributed over the cluster, and over the local
processors (or processor cores) within the single machines of the
cluster.

Medium and low level functions are also provided. These require the user
to program the scheduling of parallel execution.

Data transfer is possible between the client machine and each server
machine, but also directly between server machines. The latter feature
is exploited e.g. by the @ref{install_vars} function.

@menu
* Security::                        Security considerations.
* Authentication::                  Generating authentication keys.
* pserver::                         Starting servers.

Connection-related functions
* pconnect::                        Establishing cluster connections.
* sclose::                          Closing cluster connections.
* network_get_info::                Get network information.
* network_set::                     Set network properties.

High level functions
* netcellfun::                      Function netcellfun.
* netarrayfun::                     Function netarrayfun.
* install_vars::                    Distribute Octave variables.

Medium level functions
* rfeval::                          Single remote function evaluation.

Low level functions
* psend::                           Sending Octave variables.
* precv::                           Receiving Octave variables.
* reval::                           Remote command evaluation.
* select_sockets::                  Call Unix @code{select} for data
                                      connections.

* Limitations::                     Limitations of high- and
                                      medium-level functions.

* Example::                         Example
@end menu

@c ------------------------------------------------------------------

@node Security
@section Security considerations
@cindex security

Over a connection to a server of the parallel package arbitrary Octave
commands can be executed at the server machine. This means that any
change can be done to the server system and the resident data, only
limited by the rights of the system account under which the server runs.

By default, the server is started with authentication and encryption
enabled, to avoid unauthorized access. If the server is started with
authentication disabled (maybe to avoid the encryption overhead), it
must be cared for that no TCP connection by unauthorized persons is
possible to the server ports, possibly by running the client and all
server machines behind a firewall and assuring that only trusted persons
have access to any machine behind the firewall. This scenario might be
achievable in home-nets.

The server currently uses port 12502 for receiving commands and port
12501 for data exchange.

The client and the servers used by the client with @code{pconnect} must
agree on using authentication or not.

Do not start the server as root.

@c ------------------------------------------------------------------

@node Authentication
@section Generating authentication keys
@cindex authentication
@mfnindex parallel_generate_srp_data

@c include function helptext here
@c parallel_generate_srp_data parallel_generate_srp_data.cc
@anchor{XREFparallel_generate_srp_data}
@deftypefn {Loadable Function} {} parallel_generate_srp_data (@var{username})
@deftypefnx {Loadable Function} {} parallel_generate_srp_data (@var{username}, @var{options})
Prompts for a password (press enter for a random password) and writes TLS-SRP authentication files into the directory given by:

@code{fullfile (a = pkg ("prefix"), "parallel-srp-data")}

Server files are placed in subdirectory @code{server}. By default, a
client authentication file is placed in subdirectory
@code{client}. The latter contains the given @var{username} and the
cleartext password. You do not need this file if you prefer to be
prompted for username and password at connection time. In this case,
you can prevent the client authentication file from being written by
passing as the argument @var{options} a structure with a value of
@code{false} in the field @code{unattended}.

For authentication, subdir @code{server}, and possibly subdir
@code{client}, have to be placed together with their contents at the
respective machines. They can either be placed under the directory
given by:

@code{fullfile (OCTAVE_HOME (), "share", "octave", "parallel-srp-data")}

or -- which might be the same directory -- under:

@code{fullfile (a = pkg ("prefix"), "parallel-srp-data")}

Files in the former directory will take precedence over those in the
latter. The contents of the files @code{passwd} and @code{user_passwd}
(if present) must be kept secret.

This function zeroizes sensitive data before releasing its memory. Due
to usage of external libraries, however, it still can't be excluded
that sensitive data is still on the swap device after application
shutdown.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node pserver
@section Starting servers
@mfnindex pserver

@c include function helptext here
@c pserver pserver.cc
@anchor{XREFpserver}
@deftypefn {Loadable Function} {} pserver ()
@deftypefnx {Loadable Function} {} pserver (@var{options})
This function starts a server of the parallel cluster and should be called once at any server machine.

It is important to call this function in a way assuring that Octave is
quit as soon as the function returns, i.e. call it e.g. like
@code{octave --eval "pserver"} or @code{octave --eval "pserver
(struct ('fieldname', value))"}.

If a directory path corresponding to the current directory of the
client exists on the server machine, it will be used as the servers
current directory for the respective client (multiple clients are
possible). Otherwise, @code{/tmp} will be used. The Octave functions
the server is supposed to call and the files it possibly has to access
must be available at the server machine. This can e.g. be achieved by
having the server machine mount a network file system (which is
outside the scope of this package documentation).

The parent server process can only be terminated by sending it a
signal. The pid of this process, as long as it is running, will be
stored in the file @code{/tmp/.octave-<hostname>.pid}.

If a connection is accepted from a client, the server collects a
network identifier and the names of all server machines of the network
from the client. Then, connections are automatically established
between all machines of the network. Data exchange will be possible
between all machines (client or server) in both directions. Commands
can only be sent from the client to any server.

The opaque variable holding the network connections, in the same order
as in the corresponding variable returned by @code{pconnect}, is
accessible under the variable name @code{sockets} at the server
side. Do not overwrite or clear this variable. The own server machine
will also be contained at some index position of this variable, but
will not correspond to a real connection. See @code{pconnect} for
further information.

@var{options}: structure of options; field @code{use_tls} is
@code{true} by default (TLS with SRP authentication); if set to
@code{false}, there will be no encryption or authentication. Field
@code{auth_file} can be set to an alternative path to the file with
authentication information (see below).

The client and the server must both use or both not use TLS. If TLS is
switched off, different measures must be taken to protect ports 12501
and 12502 at the servers and the client against unauthorized access;
e.g. by a firewall or by physical isolation of the network.

For using TLS, authorization data must be present at the server
machine. These data can conveniently be generated by
@code{parallel_generate_srp_data}; the helptext of the latter function
documents the expected location of these data.

The SRP password will be sent over the encrypted TLS channel from the
client to each server, to avoid permanently storing passwords at the
server for server-to-server data connections. Due to inevitable usage
of external libraries, memory with sensitive data can, however, be on
the swap device even after shutdown of the application.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node pconnect
@section Establishing cluster connections
@mfnindex pconnect

@c include function helptext here
@c pconnect pconnect.cc
@anchor{XREFpconnect}
@deftypefn {Loadable Function} {@var{connections} =} pconnect (@var{hosts})
@deftypefnx {Loadable Function} {@var{connections} =} pconnect (@var{hosts}, @var{options})
Connects to a network of parallel cluster servers.

As a precondition, a server must have been started at each machine of
the cluster, see @code{pserver}. Connections are not guaranteed to
work if client and server are from @code{parallel} packages of
different versions, so versions should be kept equal.

@var{hosts} is a cell-array of strings, holding the names of all
server machines. The machines must be unique, and their names must be
resolvable to the correct addresses also at each server machine, not
only at the client. This means e.g. that the name @code{localhost} is
not acceptable (exception: @code{localhost} is acceptable as the first
of all names).

Alternatively, but deprecated, @var{hosts} can be given as previously,
as a character array with a machine name in each row. If it is given
in this way, the first row must contain the name of the client machine
(for backwards compatibility), so that there is one row more than the
now preferred cell-array @var{hosts} would have entries.

@code{pconnect} returns an opaque variable holding the network
connections. This variable can be indexed to obtain a subset of
connections or even a single connection. (For backwards compatibility,
a second index of @code{:} is allowed, which has no effect). At the
first index position is the client machine, so this position does not
correspond to a real connection. At the following index positions are
the server machines in the same order as specified in the cell-array
@var{hosts}. So in the whole the variable of network connections has
one position more than the number of servers given in @var{hosts}
(except if @var{hosts} was given in the above mentioned deprecated
way). You can display the variable of network connections to see what
is in it. The variable of network connections, or subsets of it, is
passed to the other functions for parallel cluster excecution
(@code{reval}, @code{psend}, @code{precv}, @code{sclose},
@code{select_sockets} among others -- see documentation of these
functions).

@var{options}: structure of options; field @code{use_tls} is
@code{true} by default (TLS with SRP authentication); if set to
@code{false}, there will be no encryption or authentication. Field
@code{password_file} can be set to an alternative path to the file
with authentication information (see below). Field @code{user} can
specify the username for authentication; if the username is so
specified, no file with authentication information will be used at the
client, but the password will be queried from the user.

The client and the server must both use or both not use TLS. If TLS is
switched off, different measures must be taken to protect ports 12501
and 12502 at the servers and the client against unauthorized access;
e.g. by a firewall or by physical isolation of the network.

For using TLS, authorization data must be present at the server
machine. These data can conveniently be generated by
@code{parallel_generate_srp_data}. By default, the client
authentication file is created in the same run. The helptext of
@code{parallel_generate_srp_data} documents the expected locations of
the authentication data.

The SRP password will be sent over the encrypted TLS channel from the
client to each server, to avoid permanently storing passwords at the
server for server-to-server data connections. Due to inevitable usage
of external libraries, memory with sensitive data can, however, be on
the swap device even after shutdown of the application, both at the
client and at the server machines.

Example (let data travel through all machines), assuming
@code{pserver} was called on each remote machine and authentication
data is present (e.g. generated with
@code{parallel_generate_srp_data}):

@example
@group
sockets = pconnect (@{'remote.machine.1', 'remote.machine.2'@});
reval ('psend (precv (sockets(2)), sockets(1))', sockets(3));
reval ('psend (precv (sockets(1)), sockets(3))', sockets(2));
psend ('some data', sockets(2));
precv (sockets(3))
--> ans = some data
sclose (sockets);
@end group
@end example

@seealso{@ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}, @ref{XREFrfeval,,rfeval}}
@end deftypefn



@c ------------------------------------------------------------------

@node sclose
@section Closing cluster connections
@mfnindex sclose

@c include function helptext here
@c sclose sclose.cc
@anchor{XREFsclose}
@deftypefn {Loadable Function} {} sclose (@var{connections})
Close the parallel cluster network to which @var{connections} belongs.

See @code{pconnect} for a description of the @var{connections}
variable. All connections of the network are closed, even if
@var{connections} contains only a subnet or a single connection.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node network_get_info
@section Get network information
@mfnindex network_get_info

@c include function helptext here
@c network_get_info network_get_info.cc
@anchor{XREFnetwork_get_info}
@deftypefn {Loadable Function} {} network_get_info (@var{connections})
Return an informational structure-array with one entry for each machine specified by @var{connections}.

This function can only be successfully called at the client
machine. See @code{pconnect} for a description of the
@var{connections} variable. @var{connections} can contain all
connections of the network, a subset of them, or a single
connection. For the local machine (client), if contained in
@var{connections}, some fields of the returned structure may be
empty.

The fields of the returned structure are @code{local_machine}: true
for the connection representing the local machine, @code{nproc}:
number of usable processors of the machine, @code{nlocaljobs}:
configured number of local processes on the machine, @code{peername}:
name of the machine (empty for local machine), @code{open}: true if
the connection is open, @code{network_id}: uuid of the network,
@code{real_node_id}: internal id assigned to node, @code{0} for
client, servers starting with @code{1}.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node network_set
@section Set network properties
@mfnindex network_set

@c include function helptext here
@c network_set network_set.cc
@anchor{XREFnetwork_set}
@deftypefn {Loadable Function} {} network_set (@var{connections}, @var{key}, @var{val})
Set the property named by @var{key} to the value @var{val} for each machine specified by @var{connections}.

This function can only be successfully called at the client
machine. See @code{pconnect} for a description of the
@var{connections} variable. @var{connections} can contain all
connections of the network, a subset of them, or a single
connection.

Possible values of @var{key}: @code{'nlocaljobs'}: configured number
of local processes on the machine (usable by functions for parallel
execution); needs a non-negative integer in @var{val}, @code{0} means
not specified.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node netcellfun
@section Function netcellfun
@mfnindex netcellfun

@c include function helptext here
@c netcellfun netcellfun.copy_to_m
@anchor{XREFnetcellfun}
@deftypefn{Function File} {} netcellfun (@var{connections}, @var{fun}, @dots{})
Evaluates function @var{fun} in a parallel cluster and collects results.

This function handles arguments and options equivalently to
@code{parcellfun} and returnes equivalent output. Differently, the
first argument specifies server machines for parallel remote
execution, see @code{pconnect} for a description of the
@var{connections} variable. A further difference is that the option
"ChunksPerProc" is ignored and instead the chunk size can be
specified directly with an option "ChunkSize".

This function can only be successfully called at the client machine
of the parallel cluster. @var{connections} can contain all
connections of the network, a subset of them, or a single connection.
The local machine (client), if contained in @var{connections}, is
ignored. However, one of the servers can run at the local machine
under certain conditions (see @code{pconnect}) and will not be
ignored in this case, so that the local machine can take part in
parallel execution with @code{netcellfun}.

As a second level of parallelism, @var{fun} is executed at each
server machine (using @code{parcellfun or pararrayfun}) by default in
as many local processes in parallel as the server has processor cores
available. The number of local parallel processes can be configured
for each server with the "nlocaljobs" option (see
@code{network_set}), a value of @code{0} means that the default value
will be used, a value of @code{1} means that execution is not
parallel within the server (but still parallel over the cluster).

There are certain limitations on how @var{fun} can be defined. These
are explained in the documentation of @code{rfeval}.

Cluster execution incurs a considerable overhead. A speedup is
likely if the computation time of @var{fun} is long. To speed up
execution of a large set of arguments with short computation times
of @var{fun}, increase "ChunkSize", possibly use "Vectorize" (see
@code{pararrayfun}), and possibly experiment with increasing
"nlocaljobs" from the default.

@seealso{@ref{XREFnetarrayfun,,netarrayfun}, @ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFsclose,,sclose}, @ref{XREFrfeval,,rfeval}, @ref{XREFinstall_vars,,install_vars}}
@end deftypefn


@c ------------------------------------------------------------------

@node netarrayfun
@section Function netarrayfun
@mfnindex netarrayfun

@c include function helptext here
@c netarrayfun netarrayfun.copy_to_m
@anchor{XREFnetarrayfun}
@deftypefn{Function File} {} netarrayfun (@var{connections}, @var{fun}, @dots{})
Evaluates function @var{fun} in a parallel cluster and collects results.

This function handles arguments and options equivalently to
@code{pararrayfun} and returnes equivalent output. Differently, the
first argument specifies server machines for parallel remote
execution, see @code{pconnect} for a description of the
@var{connections} variable. A further difference is that the option
"ChunksPerProc" is ignored and instead the chunk size can be
specified directly with an option "ChunkSize" (option "Vectorized"
can be used together with option "ChunkSize" in function
@code{netarrayfun}).

The further details of operation are the same as for
@code{netcellfun}, please see there.

@seealso{@ref{XREFnetcellfun,,netcellfun}, @ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFsclose,,sclose}, @ref{XREFrfeval,,rfeval}, @ref{XREFinstall_vars,,install_vars}}
@end deftypefn


@c ------------------------------------------------------------------

@node install_vars
@section Distribute Octave variables
@mfnindex install_vars

@c include function helptext here
@c install_vars install_vars.copy_to_m
@anchor{XREFinstall_vars}
@deftypefn{Function File} {} install_vars (@var{varname}, @dots{}, @var{connections})
Install named variables at remote machines.

The variables named in the arguments are distrubted to the remote
machines specified by @var{connections} and installed there. The
variables must be accessible within the calling function. If
variables have been declared to have global scope, they will also
have global scope at the remote machines.

This function can only be successfully called at the client machine.
See @code{pconnect} for a description of the @var{connections}
variable. @var{connections} can contain all connections of the
network, a subset of them, or a single connection. The local machine
(client), if contained in @var{connections}, is ignored.

To install, e.g., all visible variables,

@code{install_vars (who ()@{:@}, @dots{});}

can be used.

Internally, this function sends the variables only to one server and
then issues the necessary commands to distribute them to all
specified servers over server-to-server data connections.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFsclose,,sclose}, @ref{XREFrfeval,,rfeval}, @ref{XREFnetcellfun,,netcellfun}}
@end deftypefn


@c ------------------------------------------------------------------

@node rfeval
@section Single remote function evaluation
@mfnindex rfeval

@c include function helptext here
@c rfeval rfeval.copy_to_m
@anchor{XREFrfeval}
@deftypefn{Function File} {} rfeval (@var{func}, @dots{}, @var{nout}, @var{isout}, @var{connection})
Evaluate a function at a remote machine.

@var{func} is evaluated with arguments @code{@dots{}} and number of
output arguments set to @var{nout} at remote machine given by
@var{connection}. If @var{isout} is not empty, it must be a logical
array with @var{nout} elements, which are true for each of the
@var{nout} output arguments which are requested from the function;
the other output arguments will be  marked as not requested
with @code{~} at remote execution.

This function can only be successfully called at the client machine.
See @code{pconnect} for a description of the @var{connection}
variable. @var{connection} must contain one single connection.

If an output argument is given to @code{rfeval}, the function waits
for completion of the remote function call, retrieves the results and
returns them. They will be returned as one cell-array with an entry
for each output argument. If some output arguments are marked as not
requested by setting some elements of @var{isout} to false, the
returned cell-array will only have entries for the requested output
arguments. For consistency, the returned cell-array can be empty. To
assign the output arguments to single variables, you can for example
use: @code{[a, b, c] = returned_cell_array@{:@};}.

If no output argument is given to @code{rfeval}, the function does
not retrieve the results of the remote function call but returns
immediately. It is left to the user to retrieve the results with
@code{precv}. The results will be in the same format as if returned
by @code{rfeval}. Note that a cell-array, possibly empty, will always
have to be retrieved, even if the remote function call should have
been performed without output arguments.

Parallel execution can be achieved by calling @code{rfeval} several
times with different specified server machines before starting to
retrieve the results.

The specified function handle can refer to a function present at the
executing machine or be an anonymous function. In the latter case,
the function specification sent to the server includes the anonymous
functions context (generation of the sent function specification is
implemented in the Octave core). Sending a handle to a subfunction,
however, will currently not work. Sending a handle to a private
function will only work if its file path is the same at the server.
Sending an anonymous function using "varargin" in the argument list
will currently not work.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFsclose,,sclose}, @ref{XREFinstall_vars,,install_vars}, @ref{XREFnetcellfun,,netcellfun}}
@end deftypefn


@c ------------------------------------------------------------------

@node psend
@section Sending Octave variables
@mfnindex psend

@c include function helptext here
@c psend psend.cc
@anchor{XREFpsend}
@deftypefn {Loadable Function} {} psend (@var{value}, @var{connections})
Send the value in variable @var{value} to all parallel cluster machines specified by @var{connections}.

This function can be called both at the client machine and (with
@code{reval}) at a server machine. See @code{pconnect} for a
description of the @var{connections} variable, and @code{pserver} for
a description of this variable (named @code{sockets}) at the server
side. @var{connections} can contain all connections of the network, a
subset of them, or a single connection. The machine at which
@code{psend} was called (client or server), if contained in the
@var{connections} variable, is ignored.

The value sent with @code{psend} must be received with @code{precv} at
the target machine. Note that values can be sent to each machine, even
from a server machine to a different server machine.

If @code{psend} is called at the client machine, a corresponding
@code{precv} should have been called before at the target machine,
otherwise the client will hang if @var{value} contains large data
(which can not be held by the operating systems socket buffers).

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node precv
@section Receiving Octave variables
@mfnindex precv

@c include function helptext here
@c precv precv.cc
@anchor{XREFprecv}
@deftypefn {Loadable Function} {} precv (@var{single_connection})
Receive a data value from the parallel cluster machine specified by @var{single_connection}.

This function can be called both at the client machine and (with
@code{reval}) at a server machine. @var{single_connection} must be a
single connection obtained by indexing the @var{connections}
variable. Please see @code{pconnect} for a description of the
@var{connections} variable, and @code{pserver} for a description of
this variable (named @code{sockets}) at the server side. If
@var{single_connection} corresponds to the machine at which
@code{precv} was called, an error is thrown.

The value received with @code{precv} must be sent with @code{psend}
from another machine of the cluster. Note that data can be transferred
this way between each pair of machines, even sent by a server and
received by a different server.

If @code{precv} is called at the client machine, a corresponding
@code{psend} should have been called before at the source machine,
otherwise the client will hang.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node reval
@section Remote command evaluation
@mfnindex reval

@c include function helptext here
@c reval reval.cc
@anchor{XREFreval}
@deftypefn {Loadable Function} {} reval (@var{commands}, @var{connections})
Evaluate @var{commands} at all remote hosts specified by @var{connections}.

This function can only be successfully called at the client
machine. @var{commands} must be a string containing Octave commands
suitable for execution with Octaves @code{eval()} function. See
@code{pconnect} for a description of the @var{connections}
variable. @var{connections} can contain all connections of the
network, a subset of them, or a single connection. The local machine
(client), if contained in @var{connections}, is
ignored. @var{commands} is executed in the same way at each specified
machine.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}, @ref{XREFselect_sockets,,select_sockets}}
@end deftypefn



@c ------------------------------------------------------------------

@node select_sockets
@section Call Unix @code{select} for data connections
@mfnindex select sockets

@c include function helptext here
@c select_sockets select_sockets.cc
@anchor{XREFselect_sockets}
@deftypefn {Loadable Function} {} select_sockets (@var{connections}, @var{timeout})
@deftypefnx {Loadable Function} {} select_sockets (@var{connections}, @var{timeout}, @var{nfds})
Calls Unix @code{select} for data connections in a parallel cluster.

This function is for advanced usage (and therefore has minimal
documentation), typically the programming of schedulers. It can be
called at the client or at a server.

@var{connections}: valid connections object (see @code{pconnect} and
@code{pserver}, possibly indexed).

@var{timeout}: seconds, negative for infinite.

@var{nfds}: Passed to Unix @code{select} as first argument, see
documentation of Uix @code{select}. Default: @code{FD_SETSIZE}
(platform specific).

An error is returned if nfds or a watched filedescriptor plus one
exceeds FD_SETSIZE.

Returns an index vector to @var{connections} indicating connections
with pending input, readable with @code{precv}.

If called at the client, the command connections are included into the
UNIX @code{select} call and checked for error flags, and
@code{select_sockets} returns an error if a flag for a remote error is
received.

@seealso{@ref{XREFpconnect,,pconnect}, @ref{XREFpserver,,pserver}, @ref{XREFreval,,reval}, @ref{XREFpsend,,psend}, @ref{XREFprecv,,precv}, @ref{XREFsclose,,sclose}, @ref{XREFparallel_generate_srp_data,,parallel_generate_srp_data}}
@end deftypefn



@c ------------------------------------------------------------------

@node Limitations
@section Limitations of high- and medium-level functions
@cindex limitations

For the function handles passed as arguments to @ref{netcellfun},
@ref{netarrayfun}, and @ref{rfeval}, the following limitations currently
apply:

@itemize
@item If a handle to a named function is passed, this function must
exist on the server machine.
@item A handle to a subfunction can not be passed.
@item A handle to a private function can only be passed if its file path
is the same at the server.
@item If an anonymous function is passed, it may not have been defined
using @code{varargin}.
@end itemize

Anonymous functions should be usable as usual, since the function
specification sent to the server will include the anonymous functions
context.

@c ------------------------------------------------------------------

@node Example
@section Example
@cindex example

@example

# From Octave prompt, generate authentication files, set user name to
# 'test'. When prompted for a password, press <enter>.
parallel_generate_srp_data ('test')

# From Octave prompt, get location of the generated files.
authpath = fullfile (a = pkg ("prefix"), "parallel-srp-data")

Copy server files to servers, authpath is assumed to be
"/home/test/octave/parallel-srp-data/", the same directory is assumed to
exist on the servers. From the system shell, do e.g.:

scp -r /home/test/octave/parallel-srp-data/server server1:/home/test/octave/parallel-srp-data/
scp -r /home/test/octave/parallel-srp-data/server server2:/home/test/octave/parallel-srp-data/

Start server at remote machines. From the system shell, do e.g.:

ssh server1 'octave --eval "pserver"'
ssh server2 'octave --eval "pserver"'

# From Octave prompt, connect the cluster.
conns = pconnect (@{"server1", "server2"@})

# And perform some parallel execution. Single function calls take 1
# second each.
results = netcellfun (conns, @ (x) @{x, pause(1)@}@{:@}, num2cell (1:30))

# Close network.
sclose (conns)

@end example

@c ------------------------------------------------------------------

@node Further functions
@chapter Functions possibly helpful in parallel execution

@menu
* fsave fload::                 Transfer variable within the local
                                  machine over an Octave stream.
* select::                      Call Unix @code{select} on Octave
                                  streams.
@end menu

@c ------------------------------------------------------------------

@node fsave fload
@section Transfer variable within the local machine over an Octave stream
@mfnindex fsave
@mfnindex fload

@c include function helptext here
@c fsave fsave.cc
@anchor{XREFfsave}
@deftypefn {Loadable Function} {} fsave (@var{fid}, @var{var})
Save a single variable to a binary stream, to be subsequently loaded with
fload. Returns true if successful.
Not suitable for data transfer between machines of different type.
@end deftypefn



@c include function helptext here
@c fload fload.cc
@anchor{XREFfload}
@deftypefn {Loadable Function} {@var{var} =} fload (@var{fid})
Loads a single variable of any type from a binary stream, where it was previously
saved with fsave.
Not suitable for data transfer between machines of different type.
@end deftypefn



@c ------------------------------------------------------------------

@node select
@section Call Unix @code{select} on Octave streams
@mfnindex select

Note that for cluster connections @ref{select_sockets} should be used
instead.

@c include function helptext here
@c select select.cc
@anchor{XREFselect}
@deftypefn {Loadable Function} {[@var{n}, @var{ridx}, @var{widx}, @var{eidx}] =} select (@var{read_fids}, @var{write_fids}, @var{except_fids}, @var{timeout}[, @var{nfds}])
Calls Unix @code{select}, see the respective manual.

The following interface was chosen:
@var{read_fids}, @var{write_fids}, @var{except_fids}: vectors of stream-ids.
@var{timeout}: seconds, negative for infinite.
@var{nfds}: optional, default is Unix' FD_SETSIZE (platform specific).
An error is returned if nfds or a filedescriptor belonging to a stream-id
plus one exceeds FD_SETSIZE.
Return values are:
@var{n}: number of ready streams.
@var{ridx}, @var{widx}, @var{eidx}: index vectors of ready streams in
@var{read_fids}, @var{write_fids}, and @var{except_fids}, respectively.
@end deftypefn



@c ------------------------------------------------------------------

@node Documentation
@chapter Function parallel_doc to view documentation
@cindex documentation
@mfnindex parallel_doc

@c include function helptext here
@c parallel_doc ../inst/parallel_doc.m
@anchor{XREFparallel_doc}
@deftypefn {Function File} {} parallel_doc ()
@deftypefnx {Function File} {} parallel_doc (@var{keyword})
Show parallel package documentation.

Runs the info viewer Octave is configured with on the documentation
in info format of the installed parallel package. Without argument,
the top node of the documentation is displayed. With an argument,
the respective index entry is searched for and its node displayed.

@end deftypefn


@c ------------------------------------------------------------------

@node Function index
@unnumbered Index of functions in parallel

@printindex mfn

@c ------------------------------------------------------------------

@node Concept index
@unnumbered Concept index

@printindex cp

@bye
