# hydro-meteo-sim

**Repository Path**: mirrors_CiscoDevNet/hydro-meteo-sim

## Basic Information

- **Project Name**: hydro-meteo-sim
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: develop
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-12-10
- **Last Updated**: 2026-02-15

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Table of Contents

- [Sensor Protocol Simulators](#sensor-protocol-simulators)
  * [Running the Simulator](#running-the-simulator)
  * [Supported Sensor Protocols](#supported-sensor-protocols)
  * [Supported Communication Drivers](#supported-communication-drivers)
- [Package Dependencies](#package-dependencies)
- [Protocol Details](#protocol-details)
  * [CTD (Salinity)](#ctd-salinity)
  * [SIU (Waterlevel)](#siu-waterlevel)
  * [XSIAM (Wind, Visibility)](#xsiam-wind-visibility)
- [Configurations](#configurations)
  * [Settings](#settings)
- [Driver Options](#driver-options)
  * [RS232](#rs232)
  * [RS485 / RS485 Emulated](#rs485---rs485-emulated)
  * [Socket](#socket)
- [Data Options](#data-options)
  * [List](#list)
  * [BinaryFile](#binary-file)
- [Package Development](#package-development)

# Sensor Protocol Simulators

This package allows you to simulate certain sensor protocols via different communication "drivers".

## Running the Simulator

Example CLI command to run the simulator with the waterlevel configuration __from within the src directory__.

```python main.py -c configs\waterlevel_9600_8_n_1.json```

## Supported Sensor Protocols

* CTD (salinity)
    + Used for salinity sensors
* SESAM (sesam)
* SIU (siu)
    + Used for Water level sensors
* XSIAM (xsiam)
    + Used for Wind and Visibility sensors

## Supported Communication Drivers

* Console (for debugging)
* RS232
* RS485
* RS485 Emulated (For more details, visit http://pyserial.readthedocs.io/en/latest/pyserial_api.html#rs485-support)
* Socket

# Package Dependencies

* Python 2.7.3
* virtualenv
* All other dependencies can be installed by running ```pip install -r requirements.txt```
* All development dependencies (for testing) can be installed by running ```pip install -r requirements-dev.txt``` 

# Protocol details

This section explains how the different protocols spit out data. Please be aware that this represents how (raw) data is
outputted by a sensor, not how data is outputted by a potential serializer or a file based output.

## CTD (Salinity)

CTD (Conductivity, Temperature, Depth) is a simple formatting used for the salinity sensors. A message has the following
format:

```CTD
00:00:00, 31-01-2050, 00.787, 16.852, +0.00, 0.00\r\n
```

Every measurement consists of six fields:
1. Timestamp coming from sensor
2. Date coming from sensor
3. Conductivity measurement in mS/cm [Wikipedia: Siemens (unit)](<https://en.wikipedia.org/wiki/Siemens_(unit)>)
4. Temperature in degrees Celsius
5. Not relevant at the moment, but this field indicates pressure in dbars
6. Not relevent at the moment, battery voltage in volts

### Error messages

Not known at the moment

## SIU (Waterlevel)

SIU (Sensor Interface Unit) is a simple formatting used for the water level sensors. A message has the following format:
```SIU
38\r\n
```

### Error messages

1. ```No input signal```: Error returned when the SIU isn't receiving data from its sensor.
2. ```Modem not ready```: Error returned when the SIU does not see the connect modem as ready (gateway).

## XSIAM (Wind, Visibility)

The X-SIAM protocol covers a lot of different sensor types. For the HbR project, only the Wind Speed (WS), Wind 
Direction (WR), and Visibility (ZM) are used. The data format for each sensor type is roughly the same, except for 
visibility. This sensor type will have a side note.

An X-SIAM formatted message looks like this:
```X-SIAM
\x02\r\n
XEE WR16WS3FUR68US91 WR 0 0 3117 3177 3445 2869 3169 0080 00  WS 0 0 1082 1027 1152 0540 0862 0140 00  UR 0 0 3203 3177 3445 2869 3169 0080 00  US 0 0 1041 1027 1152 0540 0862 0140 00  \r\n
XEE WR16WS3FUR68US91 WR 0 0 3117 3177 3445 2869 3169 0080 00  WS 0 0 1082 1027 1152 0540 0862 0140 00  UR 0 0 3203 3177 3445 2869 3169 0080 00  US 0 0 1041 1027 1152 0540 0862 0140 00  \r\n
XEE WR16WS3FUR68US91 WR 0 0 3117 3177 3445 2869 3169 0080 00  WS 0 0 1082 1027 1152 0540 0862 0140 00  UR 0 0 3203 3177 3445 2869 3169 0080 00  US 0 0 1041 1027 1152 0540 0862 0140 00  \r\n
\x03
```

This single X-SIAM message contains three rows, each row consisting of one or multiple sensor readings. In the example 
above four different observations are returned: WR, WS, UR, and US. Each observation follows the same format. Let's have
a look at the WR observation:

```X-SIAM WR
WR 0 0 3117 3177 3445 2869 3169 0080 00  
```
Fields in this observation:
1. Unit (WR)
2. Status code (0)
3. Instrument (0)
4. Sample (3117)
5. Minute (3117)
6. Max (3445)
7. Min (2869)
8. 10 Min Avg (3169)
9. Standard Deviation (0080) <sup>1</sup>
10. Missed samples (00)

<sup>1</sup>: For visibility (ZM) this field consists out of four slashes (////) because visibility is just an observation
and not a measurement, so a standard deviation does not exist.

Measurements are repeated three times, and we only care for one of them because they should be the same. Sending the data
multiple times stems from a time where optical point-to-point equipment was used for retrieving messages. This equipment
has been replaced with radio modems (in a broad sense) but the protocol remained the same.

### Error messages

SIAM has a built in status field _status code_ in every message row that indicates the status of the connected sensor. 
This one character code is case sensitive. Some of the status codes are generic, other codes are sensor specific. We only 
focus on the generic status codes and the  sensor specific codes coming from Wind Direction (WR), Wind Speed (WS), and 
Visibility (ZM).

#### Generic status codes

* ```0```: Normal operation
* ```1```: Test operation
* ```A```: SIAM reset
* ```B```: Broken connection / No connection with sensor.
* ```C``` or ```c```: Transient error on two consecutive measurements.
* ```D``` or ```d```: Measurement is out of sensor range.
* ```E``` or ```e```: Sensor error (might be specified in sensor specific error codes).
* ```G``` or ```g```: Gain error, something is wrong with the calibration of the measurement range.
* ```H``` or ```h```: Zero point error, something is wrong with the calibration of the zero point.
* ```I```: Post reset, should only happen directly after a reset (```A```).
* ```J```: Data communication error, like checksum errors, parity errors, etc.
* ```X```: SIAM error which could indicate that the SIAM module is defective.
* ```Y```: Data unavailable, not necessarily an error as this might happen when some natural observation is blocking 
another observation. For instance, measuring a second cloud layer when the first layer is too thick.
* ```Z```: Sensor disconnected. Used for maintenance. A technician can indicate that a sensor is removed from the SIAM 
module. All data values will be filled with ```////``` when this happens.

#### Wind Direction (WR)

* ```E```: Wrong configuration of the sensor
* ```M```: General Failure bit in sensor status active
* ```N```: No data because the pole is tilted
* ```P```: No data in message (FF.FF in stead of data)
* ```Q```: Statistical Failure bit in sensor status active
* ```U```: Memory Use in sensor equals 0
* ```u```: Memory Use in sensor < 7
* ```v```: Alarm: battery power too low
* ```x```: SIAM board temperature too high (> 55°C)

#### Wind Speed (WS)

* ```D```: Range error. Speed greater than 75 m/s.
* ```E```: Wrong configuration of the sensor
* ```M```: General Failure bit in sensor status active
* ```N```: No data because the pole is tilted
* ```P```: No data in message (FF.FF in stead of data)
* ```Q```: Statistical Failure bit in sensor status active
* ```U```: Memory Use in sensor equals 0
* ```u```: Memory Use in sensor < 7
* ```v```: Alarm: battery power too low
* ```x```: SIAM board temperature too high (> 55°C)
* ```y```: Vortex compensation active (not necessarily an error)

#### Visibility (ZM)

* ```D```: Fatal range error: the measured range is smaller than 3 meter, or larger than 125 kilometer.
* ```E```: Sensor error: slashes in received data string
* ```f```: Difference error short-long basis
* ```K```: Sensor not available
* ```k```: Dirty window
* ```L```: Temperature too low
* ```M```: PSU power too high
* ```P```: Measurement cycle was interrupted.
* ```Q```: Undefined error
* ```R```: Standby
* ```S```: Lamp (light) too weak
* ```T```: Test mode

#### 

# Configurations

The simulator is configured via JSON files, located in ```src\configs\*.json```. Lets take a look at the config of the 
simulated waterlevel sensor and go over each setting to understand what it configures.

```json
{
  "name": "Virtual Water Level Sensor",
  "interval": 1.0,
  "scenarios": {
    "normal": true,
    "data_corruption": false,
    "data_empty": false,
    "data_errors": false,
    "sensor_delays": false
  },
  "simulator": {
    "type": "siu"
  },
  "driver": [
    {
      "type": "console"
    },
    {
      "type": "rs232",
      "options": {
        "port": "COM8",
        "baudrate": 9600,
        "databits": 8,
        "parity": null,
        "stopbits": 1
      }
    }
  ],
  "data": {
    "type": "list",
    "options": {
      "values": [-250, -249, -248, -247, -246, -245, -244, ...],
      "random": true
    }
  }
}
```

A config file is present for each sensor/driver combination this simulator currently supports. It would be wise to browse 
through the files in the ```src\configs``` folder to get a better understanding which sensor requires a certain protocol 
and what kind of values are used in the accompanying data. This will give a better insight  in what to expect at the 
other end of a communication channel.

## Settings

* __name__ _(str)_: Not a required property and not used by the simulator. This key was added to not depend on the name of the file
to describe what this config is trying to simulate
* __interval__ _(float)_: Intended number of seconds between a good reading. 
* __scenarios__ _(dict)_: Setting which scenarios have to be run for this sensor. Probability of a scenario is dependent on 
    the implementation of the simulated protocol. If only one scenario is enabled, the probability of that scenario equals 1. Configurability
    of these probabilities is going to be implemented in the near future. 

    __Setting all scenarios to false will result in an error!__
    
    + __normal__ _(bool)_: If the sensor should simulate normal operation i.e. spit out a value every x interval.
    + __data_corruption__ _(bool)_: If the sensor should simulate random data corruption.
    + __data_empty__ _(bool)_: If the sensor should simulate random empty data readings.
    + __data_errors__ _(bool)_: If the sensor should simulate random protocol errors.
    + __sensor_delay__ _(bool)_: If the sensor should simulate a random delay in interval.
* __simulator__ _(dict)_: What protocol simulator to use.
    + __type__ _(str)_: The type of protocol used. This can be: "siu" or "sesam" at the moment
    + __options__ _(dict)_: If necessary, options that are passed to the protocol. At the moment not used.
* __driver__  _(list)_: A list of settings used to configure which outputs are used by the simulated sensor.
    + _(dict)_: 
        + __type__ _(str)_: The driver type. This can be: "console", "rs232", "rs485", "rs485.emulated", "socket".
        + __options__ _(dict)_: Driver specific options. Fore more details see [Driver Options](#driver-options).
* __data__ _(dict)_: Configuring the data used by the simulator.
    + __type__ _(str)_: The type of data used. This can be: "list" or "sine"
    + __options__ _(dict)_: A list of options passed to the data type. These options can differ for each data type.

# Driver Options

This section explains what driver specific options can be configured in the configuration file. These options all
adhere to the __options__ key in a driver setting, as mentioned in [Configurations](#configurations).

## RS232

The RS232Driver uses pySerial for the communication via a serial port. The following defaults are used when the 
options are not set.

```json
{
  "port": "COM8",
  "baudrate": 9600,
  "databits": 8,
  "parity": null,
  "stopbits": 1,
  "timeout": 1,
  "xonxoff": false,
  "rtscts": false,
  "dsrdtr": false
}
```

You can overwrite these values if necessary via the __options__ key when using the RS232Driver.

## RS485 / RS485 Emulated

The RS485Driver/RS485EmulatedDriver use pySerial for communicating via a serial port. The following defaults are used
when options are not set.

```json
{
  "port": "COM8",
  "baudrate": 9600,
  "databits": 8,
  "parity": null,
  "stopbits": 1,
  "timeout": 1,
  "xonxoff": false,
  "rtscts": false,
  "dsrdtr": false,
  "rts_level_tx": true,
  "rts_level_rx": false,
  "loopback": false,
  "delay_before_tx": null,
  "delay_before_rx": null
}
```

You can overwrite these values if necessary via the __options__ key when using the RS485Driver/RS485EmulatedDriver.

## Socket

The SocketDriver uses TCP/IP sockets to transfer data. The following defaults are used when no options are set.

```json
{
  "host": "0.0.0.0",
  "port": 23
}
```

The host parameter defines what IP address to bind to thus declaring access for other applications. It is probably for
the best to leave this parameter as is. The port can be set to any valid integer between 1-65535 that is not in use 
by other applications on your machine. Port 23 has been chosen as the default, because that makes it easy to use a telnet
application on your localhost to access the socket's data stream. 

When running the simulator using sockets, a socket has to be opened first from a client (via a tool like PuTTY) before
the simulator resumes its operation. That is inherent to TCP/IP sockets, and the way they are implemented in this program.

Testing has shown that one simulator on Windows could only send data to three sockets at a time, so consider using 
multiple simulator instances to simulate multiple sockets. As this simulator can only simulate a single device at a 
time, it might be best to only use one socket per simulator all together, but that is up to your needs.


# Data Options

This section explains what data specific options can be configured in the configuration file. These options all
adhere to the __options__ key in a driver setting, as mentioned in [Configurations](#configurations).

## List

The List Data type (list) uses a list of values that keeps being iterated over forever. When the iterator reaches the end of the
list, the iterator index is reset to 0. The following defaults are used when no options are set for this data type. Some 
of these defaults don't make sense in simulating them and should be overriden.

```json
{
  "values": [],
  "random": false
}
```

## BinaryFile

The BinaryFile Data type (binaryfile) uses a binary file to hold the values that the simulator should output. __When using this data
type, be aware to remove the _simulator_ option from the config file!__ Failure of doing so will probably result in an
error because this would mean that the program tries to re-serialize already serialized (raw) data.

The following defaults are used when no options are set for this data type. Some 
of these defaults don't make sense in simulating them and should be overriden.

```json
{
  "file_path": null,
  "chunk_size": 1
}
```

* __file_path__ _(str)_: The path of the binary data file, either relative to where the code is run from or absolute.

* __chunk_size__ _(int)_: The size of an individual chunk of data within the binary file itself.
 
# Package Development

This package comes _pre configured_ to run your tests with nose. Please see the ```setup.cfg``` file fore more details on
what options are passed to the nose runner. In PyCharm you can configure a Nosetest runner. Via CLI you should be able to 
run ```nosetests``` from the root directory and it should work out of the box after having installed the earlier mentioned
development dependencies.