Jack's guide to data acquisition in the modern laboratory
Part 1: hardware and low-level communication
Introduction
One of the famous experiments we all learn about in school is the gold foil experiment, completed by Hans Geiger and Ernest Marsden, working under Ernest Rutherford. They bombarded gold foil with alpha particles, a small fraction of which bounced back, indicating that the foil contained very small dense positive charges. This became the discovery of the atomic nucleus. Only one in 8000 particles were strongly deflected, so Hans Geiger had to sit in a darkened room for hours on end counting flashes on a florescent screen, presumably recording his results with a pen and paper. In a modern lab, computers and automated instrumentation has relieved us of much of the drudgery of data collection, whilst also enabling new levels of repeatability in our experiments. Unfortunately, the techniques involved are not simple, and can be an intimidating mess of acronyms to a new PhD student or a researcher who is used to using highly integrated prebuilt systems and now must build a new experimental system from scratch. In this blog, I will try to explain what many of these technologies are, and how they work together. I won’t be going into them in depth, but rather giving an overview all in one place.
This blog is organised by level, starting with this page focussing on hardware, communication protocols and drivers, and the next page moving up through command sets and software libraries and finishing with programming and scripting.
This page will discuss connecting each instrument to your computer so that you can control them. Each set of hardware usually comes with some standard method for basic communication, but they vary in what is standardised and what is left for a higher level (such as your software) to deal with. Each different type of hardware also means a different port which has to be installed in the PC. Some will be there already, some won’t, some will need drivers to be installed, some won’t. Some are more reliable or faster than others.
Serial (RS232)
Left) 9-pin D-Sub socket (male), Right) 9-pin D-Sub socket (female)
This is an old standard but remains on lots of instruments where transfer speed is not critical. It usually appears as a 9-pin D-sub port (pictured), though other connectors are sometimes used. Older PCs will tend to have a serial port on the back, and on some it is present on the motherboard, just needing a cable and plug added to the case. Some newer PCs don’t have them, in which case you can use either a USB to serial converter (in which case it will act like a virtual serial port, described below) or add a PCIe serial card.
If the PC has a 9-pin D-sub with male pins, and the instrument has the female version, you can probably just connect them with a standard cable and move on. However, there are exceptions:
- Crossover or null modem cables. RS232 serial is designed so that two wires are used, and the computer (aka DTE) talks on one wire and listens on the other, while the instrument (DCE) listens on the first wire and talks on the second. Sometimes an instrument is designed as a DTE, and you’ll need to swap the wires over somewhere in-between them. Usually you can spot this as both the PC and instrument will have male pins, but not always. A cable with the wires swapped is called a crossover or null modem cable.
- Most serial ports transmit with ±12V signalling, and can receive anything between ±15V and 0V/+5V signalling. But a small number of instruments use different voltage levels. Read the manuals carefully, especially if they don’t use the usual connector.
- Most instruments just use three of the pins: TX, RX and ground. Some will also use other pins too, in which case you need a cable which has them connected. If you’re using a null modem cable, some of these pins will need to swap too.
- Occasionally, 9 pin D sub connectors can carry 3 pins for serial and use the other pins for other things, such as power supplies. This is especially likely if the device has no other connectors.
Serial is designed to send and receive bytes, but the detail of how fast they are sent and any error checking varies. This encompasses things like baud rate, byte length, parity, stop bits etc. It is rarely necessary to look at these in detail, you just copy these settings from the manual of the instrument into the configuration of whatever software you are using.
Driver support is generally excellent across Windows, Linux and Mac, with the ports detected and set up correctly when the computer is first set up.
Serial (not RS232)
When people say serial, they usually mean RS232 as described above, but there are others too. RS485 for example, allows more than one instrument to be connected to the same bus, and generally won’t be installed in a PC by default. It is more likely to appear on instruments designed for an industrial rather than lab setting. SPI and I2C are normally used inside instruments, but sometimes will be used for small devices. CAN was originally designed for use in cars, but is starting to appear on some instruments too. And some manufacturers have their own proprietary serial busses. For all of these, you can get adapters to plug into a PC. Drivers will come with the adapters, and quality varies with price.
General Purpose Instrument Bus is an older standard which also still appears on a lot of equipment. It is also sometimes called IEEE 488, though that term is also used to refer to a USBTMC feature, and certain command sets. It uses cables where the cable connector has a copy of the PC/Instrument connector on the back, so several connectors can be stacked. One controller (usually a PC) and up to 15 instruments and 20m of cables can be connected by some combination of stacking connectors on the controller and instruments and chaining cables. Each instrument needs a unique address assigned, usually through the front panel.
PCs don’t generally have a GPIB connector, but USB or PCIe adapters are available. Both adapters and cables tend to be eye-wateringly expensive, and the cables can be a pain to wrestle into place, so GPIB is rarely the first choice if an instrument has more than one option. Drivers will come with the adapter, some will offer stand-alone drivers, but many will require that you install the manufacturer’s version of VISA, which is discussed below.
Universal Serial Bus is a very versatile, fast, and ubiquitous option which is appears on most instruments which were designed in the last decade or so. Unlike other connections discussed here, it can also power some devices, and PCs typically have plenty of USB ports. Although they do come in different speeds, even the slowest ones on any post-2005 PC are much faster than serial or GPIB. It also has the advantage that most people are already familiar with them.
Three small things to watch out for:
- Cable length should be limited to 5m for reliability, though with hubs and extenders up to 25m is possible.
- Instruments can be self-powered or can draw power from the bus. Too many bus-powered devices one long cable can cause reliability issues, and unfortunately many instruments don’t say in the manual which they are. In practice this is rarely a problem, and when it is powered hubs are a good solution.
- Desktop USB ports are usually connected to ground, while laptop ones are not. Swapping between the two can leave bus powered devices with no connection to ground and have unexpected consequences for noise.
Most instruments use full size B connectors, but micro-B and mini-B are also used. If the instrument has an A connector (the type which usually appears on a PC) then it is not there for communication with a PC, it is probably there so data, settings, or screenshots can be saved directly to a USB stick.
Once the physical connections are in place, there are several methods of communication. USB devices are divided into classes, for example mice and keyboards are in the HID class. There are two classes which are commonly used for scientific instruments.
Virtual Serial
Also known as virtual COM port (VCP), Abstract Control Model (ACM), USB serial or USB communication device class (CDC). This is a class designed to work exactly like a RS232 serial port. In some cases there will be a converter and a physical serial port inside the instrument, in others it is entirely virtual, and you can also end up with virtual serial by using a USB to serial converter. If you are choosing a converter, there are differences in hardware and driver quality between cheap auction site ones and branded ones. It is worth the extra to get one from FTDI, who provide good quality drivers.
As with a serial port, you will usually have the opportunity to set baud rate and parity etc. If a converter is involved, they need to be correct. If not, they can usually be left at the defaults.
Driver support for virtual serial is a little patchy. Windows 8.1 and earlier has a driver which will only be loaded if a suitable INF file is installed, and which has various bugs which can bring your experiment to a grinding halt until the PC is rebooted. Windows 10 contains a better driver, which will be loaded automatically when most converters and devices are plugged in. Many manufacturers (Razorbill Instruments included) will also offer their own driver which will be more reliable than either Windows driver, and may also help identify the instrument in windows dialogs. Whichever driver you use, a new port will appear, just as though a new physical serial port had been installed, and it will be allocated a name like COM1. A minor annoyance is that plugging the same instrument into a different physical USB port will sometimes result in a new and different name, so one can quickly end up with a confusing array of COM2, COM8, COM26 etc. and no idea which instrument is which.
USBTMC
The USB Test and Measurement Class is specifically designed for instruments, and is very similar to virtual serial but better. The trappings of serial such as baud rate settings are gone. There is also a subclass called 488 which adds some nice features for when timing is important. Unfortunately, there is no driver in Windows by default. Manufacturers often don’t provide stand-alone drivers, and although a generic driver does exist on the internet, installing it is tricky. Most of the time, you will be best off installing a VISA implementation (discussed in the next part), from the instrument manufacturer or elsewhere, in which case that comes with a good driver.
Ethernet
Ethernet is another common and familiar bus which has been co-opted for use on scientific instruments. Instruments can be connected with normal cables and switches designed for networking.
In a typical lab setting, it may be possible to just plug all your instruments into the Ethernet ports managed by your local IT department, but in many cases this will cause headaches for both you and IT. An alternative is to add a second Ethernet card into your PC and have two separate networks. As long as you choose your IP addresses correctly, your PC will automatically send the right traffic to the right network.
In general, if you are making your own network, you should use IP addresses from the ranges set aside for private networks, 192.168.0.[1 to 254] is a good choice in most cases, though if your IT department is using that, you should pick something else. If you install a DHCP server on your PC, most instruments will get an IP address automatically, or if you prefer you can go around setting them manually.
When connecting to an ethernet instrument from a PC, you will need to specify a port number, which you will find in the instrument manual or settings. You can use two different protocols to talk to the instruments: TCP and UDP. UDP is usually a little faster, but messages can go missing or arrive out of order, at least in theory (though probably not often unless there are a lot of other things happening on the network). TCP prevents this but is a little slower.
In addition to allowing communication through TCP/UDP, many instruments with Ethernet also have miniature web servers embedded in them. You can enter the IP address of the instrument into the address bar of your browser and see a control panel for the instrument. This is particularly popular with instruments where the physical front panel is removed or simplified to reduce costs.
EtherCat
Ethercat is based on Ethernet, but has been tweaked to give lower latency, and is valuable when sub-millisecond timing is important. With EtherCat, you generally cannot use normal Ethernet switches, instead each instrument has two or more ports, and you build a network by daisy chaining them. Many but not all EtherCat instruments also support Ethernet as above. EtherCat tends to be an expensive add-on, so is less common than ordinary Ethernet.
Others
While the hardware communication types described above are the most common, there are a few others you might encounter. PXI and VXI are both extensions of busses normally used inside PCs. PXI is an extension of PCI, and VXI is an extension of the older VMEBus. Both are commonly used to build composite instruments from a selection of individual boards, and don’t appear often in a typical lab setting. LXI works in a similar way, but uses Ethernet.
Once you have all the hardware set up, you can move onto software on the PC which will be controlling the measurement. This is covered in the next section: Part 2: IVI, VISA, and scripting to communicate with lab instruments.