Keeping it safe

Safety

For simplicity and cost, the camera system was designed with no dome or cover. The camera, lens, motorized mount and electronics (except the laptop computer) are outside, exposed to large temperature changes, wind, rain, snow and sunlight. The hardware and software were designed to operate in this environment, and much attention was paid to avoiding failures and bugs that could destroy hardware :

The camera must never point at the Sun: with a 70mm diameter aperture, this would destroy the camera and probably the lens as well
The camera must not point up when raining
The mount should not be allowed to rotate past its limits, as it would destroy something (not sure what... I've never dared to try)

Hardware safety

Protecting the camera, mount and electronics

The camera body and mount are sealed using layers of plastic film, aluminum tape and weather-proof silicone to keep water away from electronics in the mount and camera body.
Most of the electronics is mounted on an aluminum plate just under the main plate supporting the mount and cylinder. A rubber foam tube is wrapped around the lower plate, making a seal with the upper plate to keep water from entering the electronics. The few connectors which are exposed to the outside are sealed with Silicone to keep moisture out.

Power failures

The full system is on a dedicated UPS, which provides about 2 hrs of power in case of a power failure. An AC to DC converter upstream of the UPS produces an analog signal (nominally about 4.5V) which is monitored by the electronics (using an analog to digital converter). If the system detects that power is out, the camera is parked and waits until power is back up to resume observations. This ensures that if a long duration power failure occurs, the camera is not left pointing up when the UPS runs out of power.
Power to the mount and camera is turned off to extend UPS battery time in case of a long (> few minutes) power failure.

Mount

The mount used for the project does not come with limit switches, so I had to mount mechanical limit switched to both axes of the mount. The limit switches are read by the computer through a digital I/O board. When homing the mount, a slow slew command is issued, and only interrupted when the mount hits the limit switch(es). Under this scheme, if a hardware failure occurs on a limit switch, or if the computer crashes during the homing routine, the mount will go past the limit switch and break something. To prevent this, a second set of limit switched is installed past the first set, and this second set is hardwired to the power of the mount: if the mount gets to one of these limit switches, it will loose power and stop.
If that happens, it is possible to recover the mount remotely (but not automatically) thanks to a relays that bypass the limit switches to power back up the mount, and then issue commands to move the mount out of these limit switches. This is done while checking the webcams for mount motion, and it is essential to watch the webcam in order to know which way the mount should be moved, and how to move it back to a safe position (parked). This happened once so far: due to a software error, the mount RA axis went into the second limit switch. The mount was recovered remotely with the steps descibed above.

Software safety

Weather

Weather conditions are read from multiple sources:

Sensors on the unit: humidity, temperature (2 sensors, differential measurement detects low-level clouds), light (3 webcams)
VYSOS sensors: humidity, wind, rain, cloud sensor
MLO tower weather sensors

If any of the sensors shows high humidity of rain, the camera is parked. The camera can operate if communication to outside sensors is lost, in which case is relies entirely on the sensors on the unit. If the humidity or light sensors on the unit are not returning proper signal, the camera is parked.

Power and heartbeat

As described in the hardware section above, an analog signal is monitored to check that AC power is getting into the UPS. If the power goes down, the system is parked and partially powered off (power to laptop and essential electronics left on).
When executing the main loop, the software regularly updates a file ("touches it" with the unix touch command). A separate monitoring program is checking that this hearbeat is beating. If the main program crashes, the heartbeat will stop and the monitoring program takes control of the mount to park it into a safe position, and then terminates. This leaves the mount in a safe position with neither program running.

Safe mode in software

If the main loop detects an abnormal behavior, the system enters safe mode: the mount is parked and the program is terminated. The failures or behaviors that trigger safe mode include:

Malfunction in the humidity sensor
Failure to read or write to essential files on disk
Timeout while waiting for an exposure to be be completed (camera failure)
Large number of consecutive failures to talk to a device (communication failure)
Internal software bug: the software is written to continously track which file descriptors are open and closed, and if it detects a mismatch (for example, and attempt to open a file which is already open), it will enter safe more and exit. This prevents having coding errors produce unforeseen and potentially dangerous results.

Page content last updated: 27/06/2023 06:35:52 HST
html file generated 27/06/2023 06:34:44 HST