Server application

This section provides a description and reference of all functions and objects used in the main DAQBroker server application. This application consists of several subprocesses, each responsible to ensure that the relevant services and automatic tasks are preformed

Monitoring Server

Description

The monitoring server is the workhorse of DAQBroker. It is responsible for recognizing all the database engines users have connected to, gathering the state of each database associated with each engine and collecting gathering and storing the most recent data from all instruments associated with each database. The following is a list of all objects and functions associated with the monitoring server.

Reference

monitorServer.backingUp(serverObject, metaid, database, instrument, paths, server)

Auxiliary function used by the singleMonitor() function to update the database state when a remote instrument’s data is being requested

Parameters:
  • serverObject – (serverData) Object containing the current database state
  • metaid – (Integer) unique data source identifier
  • database – (String) database name
  • instrument – (String) instrument name
  • paths – (Dict) contains local storage paths
  • server – (String) database engine server name
monitorServer.checkAddresses(nodes)

Function periodically called by producer(). This function checks the nodes variable and the locally stored entries of network DAQBroker client applications. It updates the process-shared variable with new nodes to make sure newly added nodes can be seamlessly used for instrument monitoring

Parameters:nodes – (multiporcessing.Manager().list) Contains a process-shared list of existing and active DAQBroker

client applications connected to the server.

monitorServer.checkServers(servers, children, sendBack, logPort, isRemote, nodes, paths, backupInfo)

Function periodically called by producer(). This function checks the servers variable and the existing instances of singleMonitor() functions for a server unassigned to a monitor and if so starts a new instance of singleMonitor() for said server. This function keeps all database servers supplied to DAQBroker updated

Parameters:servers – (multiporcessing.Manager().list) Contains a process-shared list of database servers under

monitoring by DAQBroker :param children: (List) list of objects defining the existing singleMonitor() processes associated with each database server. :param sendBack: (Integer) network communications port. This value is sent to all remote DAQBroker clients to allow responses to be sent to the appropriate port :param logPort: (Integer) local event logging port :param isRemote: (Boolean) decides whether messages are sent to remote clients :param nodes: (multiporcessing.Manager().list) Contains a process-shared list of existing and active DAQBroker client applications connected to the server. :param paths: (dict) contains the available DAQBroker paths as stored by the users on the local settings file

monitorServer.checkTime(context, logPort)
Auxiliary function that updates the current machine time with an NTP server (if applicable) and updates the local
settings file with the most recent local machine information
Parameters:context – (zmq.context) context object to create network communications. Only used in this case for local

communication with the event logging server :param logPort: (Integer) local event logging port

monitorServer.getLocalTime(timeback)

Auxiliary function used by the getPortData() function to collect time information from an NTP port

Parameters:timeback – (multiprocessing.Value) process-shared double, this value is set with the local time
monitorServer.getMachineDetails(extra)

Auxiliary function used by the checkTime() and checkAddresses() functions to collect local machine information and synchronize machine time (if applicable)

Parameters:extra – (Dict) NTP server information
Returns:(Dict) containing the local machine information with CPU, ROM and RAM information
monitorServer.getNTPTIme(server, timeback)

Auxiliary function used by the getPortData() function to collect time information from an NTP port

Parameters:
  • server – Dict with NTP server information
  • timeback – (multiprocessing.Value) process-shared double, this value is set with the NTP time
monitorServer.getPeripheralData(database, instrument, meta, type, backupPort, backupUser, backupPass, metaName, metaid, sendBackPort, serverDB, engineDB, channels)

Auxiliary function used by the singleMonitor() function to collect data from a local peripheral instrument (COM port or network port)

Parameters:
  • database – (String) database name
  • instrument – (String) instrument name
  • meta – (Dict) contains the relevant information for the data source
  • type – (Integer) type of peripheral to obtain data from
  • backupPort – (Integer) a stored data backup network server port
  • backupUser – (String) a stored data backup server username
  • backupPass – (String) a stored data backup server password
  • metaName – (String) data source name
  • metaid – (Integer) unique data source identifier
  • sendBackPort – (Integer) network communications port
  • serverDB – (String) database engine server address
  • engineDB – (String) database engine
  • channels – (List) list of data channel objects
monitorServer.producer(servers, sendBack, logPort, isRemote, backupInfo, workerList, localPath)

Main monitoring process loop. This loop is responsible for persistently checking the available DAQBroker servers and creating a new server monitoring process (see monitorServer.singleMonitor) in case a new server is provided by a user.

Parameters:servers – (multiporcessing.Manager().list) Contains a process-shared list of database servers under

monitoring by DAQBroker :param sendBack: (Integer) network communications port. This value is sent to all remote DAQBroker clients to allow responses to be sent to the appropriate port :param logPort: (Integer) local event logging port :param isRemote: (Boolean) decides whether messages are sent to remote clients :param backupInfo: (multiporcessing.Manager().list) process-shared dict with relevant backup information

Warning

This is a long running process and blocks execution of the main task, it should therefore be called on a separate process.

class monitorServer.serverData(server='localhost', engine='mysql', user='username', pword='password')

Class containing the relevant monitoring information on a single database engine. This class includes the following members

Variables:
  • server – (string) server address
  • engine – (string) server database engine
  • password – (string) database password
  • username – (string) database username
  • databases – (dict) database information dict object. This object is
  • db – (sqlalchemy.engine.connection) database connection object
refreshGlobals()

Updates the database engine state. Goes through all DAQBroker databases, instruments, data sources and channels and gathers the most recent monitoring and data gathering information, updating the databases attribute

monitorServer.singleMonitor(server, engine, username, password, sendBack, logPort, isRemote, nodes, paths, backupInfo)
Single database engine monitoring function. This function is responsible for continuously updating the status of
a database engine and triggering the automated DAQBroker monitoring mechanisms. It periodically runs several housekeeping tasks on the global database settings but most importantly it constantly retrieves instrument data from an internal serverData object and decides appropriately what action must be taken for each instrument in each database.
Parameters:
  • server – (String) database server address
  • engine – (String) database server engine
  • username – (String) database server username
  • password – (String) database server password
  • sendBack – (Integer) network communications port. This value is sent to all remote DAQBroker clients to allow responses to be sent to the appropriate port
  • logPort – (Integer) local event logging port
  • isRemote – (Boolean) decides whether messages are sent to remote clients
  • nodes – (multiporcessing.Manager().list) Contains a process-shared list of existing and active DAQBroker

client applications connected to the server. :param paths: (dict) contains the available DAQBroker paths as stored by the users on the local settings file

Warning

This is a long running process and blocks execution of the main task, it should therefore be called on a separate process.

monitorServer.syncInst(sendBackPort, instMeta, instrument, metaid, database, logPort, backupPort, backupUser, backupPass, serverDB, engineDB, metaName)

Auxiliary function used by the singleMonitor() function to collect data from a local file-based instrument

Parameters:
  • sendBackPort – (Integer) local communications port
  • instMeta – (Dict) information on the data source
  • instrument – (String) instrument name
  • metaid – (Integer) unique data source identifier
  • database – (String) database name
  • logPort – (Integer) local event logging port
  • backupPort – (Integer) a stored data backup network server port
  • backupUser – (String) a stored data backup server username
  • backupPass – (String) a stored data backup server password
  • serverDB – (String) database engine server address
  • engineDB – (String) database engine
  • metaName – (String) data source name

Backup Server

Description

The backup server is responsible for ensuring that the relevant data from each instrument is properly gathered and updated in a central folder structure. The following is a list of all objects and functions associated with the backup server.

Reference

backupServer.convertFilePath(path)

Auxiliary function, used only in the case the opearating system is windows. Converts Windows style paths to their cygpath equivalent

Parameters:path – (String) path to be changed
Returns:(String) cygpath-style path
backupServer.startBackup(path, backupInfo, localPath)

Backup server main process. This process is responsible for gathering data files from remote DAQBroker clients and ensuring they are at their most recent versions. This process starts an rsync daemon instance specifically tailored for DAQBroker’s needs.

Parameters:
  • path – (String) path of the current backups folder
  • backupInfo – (multiporcessing.Manager().list) process-shared dict with relevant backup information
Returns:

False in case a problem occurs

Warning

This is a long running process and blocks execution of the main task, it should therefore be called on a separate process.

Communications Server

Description

The communications server allows remote machines to connect to a central DAQBroker server application and send instrument information from instruments that would otherwise be out of reach from the main server application. The following is a list of all objects and functions associated with the communications server.

Reference

commServer.backupOver(scopedSession, server, database, metaid, instrument, logPort, lockList, paths)

Supporting function that updates the state of the database when a remote instrument’s data backup is completed

Parameters:
  • server – (Dict) server dictionary, contains the address and the database engine
  • database – (String) database name
  • metaid – (Integer) unique data source identifier
  • instrument – (String) instrument name
  • logPort – (String) database server address
  • lockList – (String) database server address
  • paths – (multiporcessing.Manager().list) database server address
commServer.collector(servers, port, logPort, backupInfo, localPath)

Communications server main process loop. This process is responsible for listening for inbound DAQBroker client communications and handling the sent requests. Each client request will have a specific node identifier associated with it as well as an order to be fulfilled.

Parameters:
  • servers – (multiporcessing.Manager().list) process-shared list of database servers under monitoring by DAQBroker. They are used here to update the state of instruments in the databases
  • port – (Integer) Port for network communications
  • logPort – (Integer) The local event logging port. See logServer for more information
  • backupInfo – (multiporcessing.Manager().list) process-shared dict with relevant backup information

Warning

This is a long running process and blocks execution of the main task, it should therefore be called on a separate process.

Event Log Server

Description

The event server is a background process that is responsible for creating an ordered list of events that occurr in the DAQBroker framework. An event can be of one of three types:

  • Information - information on an event that should regularly occur.
  • Warning - information on possible exceptions that don’t break the framework flow
  • Error - information on unexpected exceptions that break the framework flow

Each event is also associated with a specific subprocess or function on which it occurred. The following is a list of all objects and functions associated with the logging server.

Reference

logServer.logServer(port, base_dir, logFilename='logFile.log')

Logging server process, This process creates a local listening port that receives requests for event logging.

Parameters:
  • port – (Integer) network port to listen for event logging requests
  • base_dir – (String) path of the executable being called. This is for the use of a frozen function
  • logFilename – (String) name of logFile. Defaults to logFile.log

Warning

This is a long running process and blocks execution of the main task, it should therefore be called on a separate process.