4. Serial Port and Modem Basics
You don't have to understand the basics to use and install a modem. But understanding it may help to determine what is wrong if you run into problems. After reading this section, if you want to understand it even better you may want to see How Modems Work in this document (not yet complete). More details on the serial port (including much of this section) will be found in Serial-HOWTO.
4.1 Modem Converts Digital to Analog (and conversely)
Most all telephone main lines are digital already but the lines leading to your house (or business) are usually analog which means that they were designed to transmit a voltage wave which is an exact replica of the sound wave coming out of your mouth. Such a voltage wave is called "analog". If viewed on an oscilloscope it looks like a sine wave of varying frequency and amplitude. A digital signal is like a square wave. For example 3 v (volts) might be a 1-bit and 0 v could be a 0-bit. For most serial ports (used by external modems) +12 v is a 0-bit and -12 v is a 1-bit (some are + or - 5 v).
To send data from your computer over the phone line, the modem takes the digital signal from your computer and converts it to "analog". It does this by both creating an analog sine wave and then "MODulating" it. Since the result still represents digital data, it could also be called a digital signal instead of analog. But it looks something like an analog signal and almost everyone calls it analog. At the other end of the phone line another modem "DEModulates" this signal and the pure digital signal is recovered. Put together the "mod" and "dem" parts of the two words above and you get "modem" (if you drop one of the two d's). A "modem" is thus a MODulator-DEModulator. Just what modulation is may be found in the section Modulation Details.
4.2 What is a Serial Port ?
Intro to Serial
The UART serial port (or just "serial port for short" is an I/O (Input/Output) device. Since modems have a serial port between them and the computer, it's necessary to understand the serial port as well as the modem.
Most PC's have one or two serial ports. Each has a 9-pin connector (sometimes 25-pin) on the back of the computer. Computer programs can send data (bytes) to the transmit pin (output) and receive bytes from the receive pin (input). The other pins are for control purposes and ground.
The serial port is much more than just a connector. It converts the data from parallel to serial and changes the electrical representation of the data. Inside the computer, data bits flow in parallel (using many wires at the same time). Serial flow is a stream of bits over a single wire (such as on the transmit or receive pin of the serial connector). For the serial port to create such a flow, it must convert data from parallel (inside the computer) to serial on the transmit pin (and conversely).
Most of the electronics of the serial port is found in a computer chip (or a part of a chip) known as a UART. For more details on UARTs see the section "What are UARTS" in the Serial-HOWTO. But you may want to finish this section first so that you will hopefully understand how the UART fits into the overall scheme of things.
Pins and Wires
Old PC's used 25 pin connectors but only about 9 pins were actually used so today most connectors are only 9-pin. Each of the 9 pins usually connects to a wire. Besides the two wires used for transmitting and receiving data, another pin (wire) is signal ground. The voltage on any wire is measured with respect to this ground. Thus the minimum number of wires to use for 2-way transmission of data is 3. Except that it has been known to work with no signal ground wire but with degraded performance and sometimes with errors.
There are still more wires which are for control purposes (signalling) only and not for sending bytes. All of these signals could have been shared on a single wire, but instead, there is a separate dedicated wire for every type of signal. Some (or all) of these control wires are called "modem control lines". Modem control wires are either in the asserted state (on) of +12 volts or in the negated state (off) of -12 volts. One of these wires is to signal the computer to stop sending bytes out the serial port cable. Conversely, another wire signals the device attached to the serial port to stop sending bytes to the computer. If the attached device is a modem, other wires may tell the modem to hang up the telephone line or tell the computer that a connection has been made or that the telephone line is ringing (someone is attempting to call in). See the Serial-HOWTO: Pinout and Signals for more details.
Internal Modem Contains Serial Port
For an internal modem there is no 9-pin connector but the behavior is almost exactly as if the above mentioned cable wires existed. Instead of a a 12 volt signal in a wire giving the state of a modem control line, the internal modem may just use a status bit in its own memory (a register) to determine the state of this non-existent "wire". The internal modem's serial port looks just like a real serial port to the computer. It even includes the speed limits that one may set at ordinary serial ports such as 115200 bits/sec.
Unfortunately for Linux, many internal modems today use software (running on the CPU) to do much of the modem's work. Unfortunately, such software may be only available for the MS Windows OS (it hasn't been ported to Linux). Thus you can't use some of these modems with Linux See Software-based Modems (winmodems).
4.3 IO Address & IRQ
Since the computer needs to communicate with each serial port, the operating system must know that each serial port exists and where it is (its I/O address). It also needs to know which wire (IRQ number) the serial port must use to request service from the computer's CPU. It requests service by sending an interrupt voltage on this wire. Thus every serial port device must store in its non-volatile memory both its I/O address and its Interrupt ReQuest number: IRQ. See Interrupts. The PCI bus has its own system of interrupts. But since the PCI-aware BIOS sets up these PCI interrupts to map to IRQs, it seemingly behaves just as described above. Except that sharing of PCI interrupts is allowed (2 or more devices may use the same IRQ number).
I/O addresses are not the same as memory addresses. When an I/O addresses is put onto the computer's address bus, another wire is energized. This both tells main memory to ignore the address and tells all devices which have I/O addresses (such as the serial port) to listen to the address sent on the bus to see if it matches the device's. If the address matches, then the I/O device reads the data on the data bus.
The I/O address of a certain device (such as ttyS2) will actually be a range of addresses. The lower address in this range is the base address. "address" usually means just the "base address".
4.4 Names: ttyS0, ttyS1, etc.
The serial ports are named ttyS0, ttyS1, etc. (and usually correspond respectively to COM1, COM2, etc. in DOS/Windows). The /dev directory has a special file for each port. Type "ls /dev/ttyS*" to see them. Just because there may be (for example) a ttyS3 file, doesn't necessarily mean that there exists a physical serial port there.
Which one of these names (ttyS0, ttyS1, etc.) refers to which
physical serial port is determined as follows. The serial driver
(software) maintains a table showing which I/O address corresponds to
which ttyS. This mapping of names (such as ttyS1) to I/O addresses
(and IRQ's) may be both set and viewed by the "setserial" command.
See
What is Setserial. This does
not set the I/O address and IRQ in the hardware itself (which is
set by jumpers or by plug-and-play software). Thus which physical port
corresponds to say ttyS1 depends both on what the serial driver thinks
(per setserial) and what is set in the hardware. If a mistake has
been made, the physical port may not correspond to any name (such as
ttyS2) and thus it can't be used. See
Serial Port Devices /dev/ttyS2, etc. for more details.
4.5 Interrupts
Bytes come in over the phone line to the modem, are converted from analog to digital by the modem and passed along to the serial port on their way to their destination inside your computer. When the serial port receives a number of bytes (may be set to 1, 4, 8, or 14) into its FIFO buffer, it signals the CPU to fetch them by sending an electrical signal known as an interrupt on a certain wire normally used only by that port. Thus the FIFO waits until it has received a number of bytes and then issues an interrupt.
However, this interrupt will also be sent if there is an unexpected delay while waiting for the next byte to arrive (known as a timeout). Thus if the bytes are being received slowly (such as from someone typing on a terminal keyboard) there may be an interrupt issued for every byte received. For some UART chips the rule is like this: If 4 bytes in a row could have been received in an interval of time, but none of these 4 show up, then the port gives up waiting for more bytes and issues an interrupt to fetch the bytes currently in the FIFO. Of course, if the FIFO is empty, no interrupt will be issued.
Each interrupt conductor (inside the computer) has a number (IRQ) and the serial port must know which conductor to use to signal on. For example, ttyS0 normally uses IRQ number 4 known as IRQ4 (or IRQ 4). A list of them and more will be found in "man setserial" (search for "Configuring Serial Ports"). Interrupts are issued whenever the serial port needs to get the CPU's attention. It's important to do this in a timely manner since the buffer inside the serial port can hold only 16 incoming bytes. If the CPU fails to remove such received bytes promptly, then there will not be any space left for any more incoming bytes and the small buffer may overflow (overrun) resulting in a loss of data bytes.
For an external modem, there may be no way (such as flow control) to stop the flow rapidly enough to prevent such an overrun. For an internal modem, the 16-byte FIFO buffer is on the same card and most modems will not write to it if it's full. Thus it will not overrun the 16-byte buffers but it may need to use Modem-to-Modem Flow Control to prevent the modem itself from being overrun. This is one advantage of internal modems over an external.
Interrupts are also issued when the serial port has just sent out all of its bytes from its small transmit FIFO buffer out the external cable. It then has space for 16 more outgoing bytes. The interrupt is to notify the CPU of that fact so that it may put more bytes in the small transmit buffer to be transmitted. Also, when a modem control line changes state, an interrupt is issued.
The buffers mentioned above are all hardware buffers. The serial port also has large buffers in main memory. This will be explained later
Interrupts convey a lot of information but only indirectly. The interrupt itself just tells a chip called the interrupt controller that a certain serial port needs attention. The interrupt controller then signals the CPU. The CPU then runs a special program to service the serial port. That program is called an interrupt service routine (part of the serial driver software). It tries to find out what has happened at the serial port and then deals with the problem such a transferring bytes from (or to) the serial port's hardware buffer. This program can easily find out what has happened since the serial port has registers at IO addresses known to the serial driver software. These registers contain status information about the serial port. The software reads these registers and by inspecting the contents, finds out what has happened and takes appropriate action.
4.6 Data Compression (by the Modem)
Before continuing with the basics of the serial port, one needs to understand about something done by the modem: data compression. In some cases this task is actually done by software run on the computer's CPU but unfortunately at present, such software only works for MS Windows. The discussion here will be for the case where the modem itself does the compression since this is what must happen in order for the modem to work under Linux.
In order to send data faster over the phone line, one may compress (encode it) using a custom encoding scheme which itself depends on the data. The encoded data is smaller than the original (less bytes) and can be sent over the Internet in less time. This process is called "data compression".
If you download files from the Internet, they are likely already compressed and it is not feasible for the modem to try to compress them further. Your modem may sense that what is passing thru has already been compressed and refrain from trying a compress it any more. If you are receiving data which has been compressed by the other modem, your modem will decompress it and create many more bytes than were sent over the phone line. Thus the flow of data from your modem into your computer will be higher than the flow over the phone line to you. The ratio of this flow is called the compression ratio. Compression ratios as high as 4 are possible, but not very likely.
4.7 Error Correction
Similar to data compression, modems may be set to do error correction. While there is some overhead cost involved which slows down the byte/sec flow rate, the fact that error correction strips off start and stop bits actually increases the data byte/sec flow rate.
For the serial port's interface with the external world, each 8-bit byte has 2 extra bits added to it: a start-bit and a stop-bit. Without error correction, these extra start and stop bits usually go right thru the modem and out over the phone lines. But when error correction is enabled, these extra bits are stripped off and the 8-bit bytes are put into packets. This is more efficient and results in higher byte/sec flow in spite of the fact that there are a few more bytes added for packet headers and error correction purposes.
4.8 Data Flow (Speeds)
Data (bytes representing letters, pictures, etc.) flows from your computer to your modem and then out on the telephone line (and conversely). Flow rates (such as 56k (56000) bits/sec) are (incorrectly) called "speed". But almost everyone says "speed" instead of "flow rate". If there were no data compression the flow rate from the computer to the modem would be about the same as the flow rate over the telephone line.
Actually there are two different speeds to consider at your end of the phone line:
- The speed on the phone line itself (DCE speed) modem-to-modem
- The speed from your computer's serial port to your modem (DTE speed)
When you dial out and connect to another modem on the other end of the phone line, your modem often sends you a message like "CONNECT 28800" or "CONNECT 115200". What do these mean? Well, its either the DCE speed or the DTE speed. If it's higher than the advertised modem speed it must be the DTE modem-to-computer speed. This is the case for the 115200 speed shown above. The 28800 must be a DCE (modem-to-modem) speed since the serial port has no such speed. One may configure the modem to report either speed. Some modems report both speeds and report the modem-to-modem speed as (for example): CARRIER 28800.
If you have an internal modem you would not expect that there would be any speed limit on the DTE speed from your modem to your computer since you modem is inside your computer and is almost part of your computer. But there usually is since the modem contains a dedicated serial port within it. On some software modems there is no such speed limit.
It's important to understand that the average speed is often less than the specified speed, especially on the short DTE computer-to-modem line. Waits (or idle time) result in a lower average speed. These waits may include long waits of perhaps a second due to Flow Control. At the other extreme there may be very short waits (idle time) of several micro-seconds separating the end of one byte and the start of the next byte. In addition, modems will fallback to lower speeds if the telephone line conditions are less than pristine.
For a discussion of what DTE speed is best to use see section What Speed Should I Use.
4.9 Flow Control
Flow control means the ability to slow down the flow of bytes in a wire. For serial ports this means the ability to stop and then restart the flow without any loss of bytes. Flow control is needed for modems and other hardware to allow a jump in instantaneous flow rates.
Example of Flow Control
For example, consider the case where you connect a 33.6k external modem via a short cable to your serial port. The modem sends and receives bytes over the phone line at 33.6k bits per second (bps). Assume it's not doing any data compression or error correction. You have set the serial port speed to 115,200 bits/sec (bps), and you are sending data from your computer to the phone line. Then the flow from the your computer to your modem over the short cable is at 115.2k bps. However the flow from your modem out the phone line is only 33.6k bps. Since a faster flow (115.2k) is going into your modem than is coming out of it, the modem is storing the excess flow (115.2k -33.6k = 81.6k bps) in one of its buffers. This buffer would soon overrun (run out of free storage space) unless the high 115.2k flow is stopped.
But now flow control comes to the rescue. When the modem's buffer is almost full, the modem sends a stop signal to the serial port. The serial port passes on the stop signal on to the device driver and the 115.2k bps flow is halted. Then the modem continues to send out data at 33.6k bps drawing on the data it previous accumulated in its buffer. Since nothing is coming into this buffer, the number of bytes in it starts to drop. When almost no bytes are left in the buffer, the modem sends a start signal to the serial port and the 115.2k flow from the computer to the modem resumes. In effect, flow control creates an average flow rate in the short cable (in this case 33.6k) which is significantly less than the "on" flow rate of 115.2k bps. This is "start-stop" flow control.
In the above simple example it was assumed that the modem did no data compression. This could happen when the modem is sending a file which is already compressed and can't be compressed further. Now let's consider the opposite extreme where the modem is compressing the data with a high compression ratio. In such a case the modem might need an input flow rate of say 115.2k bps to provide an output (to the phone line) of 33.6k bps (compressed data). This compression ratio is 3.43 (115.2/33.6). In this case the modem is able to compress the 115.2 bps PC-to-modem flow and send the same data (in compressed form) out the phone line at 33.6bps. There's no need for flow control here so long as the compression ratio remains higher than 3.43. But the compression ratio varies from second to second and if it should drop below 3.43, flow control will be needed
In the above example, the modem was an external modem. But the same situation exists (as of early 2003) for most internal modems. There is still a speed limit on the PC-to-modem speed even though this flow doesn't take place over an external cable. This makes the internal modems compatible with the external modems.
In the above example of flow control, the flow was from the computer to a modem. But there is also flow control which is used for the opposite direction of flow: from a modem (or other device) to a computer. Each direction of flow involves 3 buffers: 1. in the modem 2. in the UART chip (called FIFOs) and 3. in main memory managed by the serial driver. Flow control protects all buffers (except the FIFOs) from overflowing.
Under Linux, the small UART FIFO buffers are not protected by flow control but instead rely on a fast response to the interrupts they issue. Some UART chips can be set to do hardware flow control to protect their FIFOs but Linux (as of early 2003) doesn't seem to support it. FIFO stand for "First In, First Out" which is the way it handles bytes in a queue. All the 3 buffers use the FIFO rule but only the one in the UART is named "FIFO". This is the essence of flow control but there are still some more details.
If you have set the highest PC-to-modem speed, you may not need flow control in the direction from the modem to a PC. For a complex example of a case where it's needed see "Complex Flow Control Example" in the Serial-HOWTO. To slow down this incoming flow, your modem must tell the other modem to stop sending. See M-M_flow_c name="Modem-to-Modem Flow Control">.
Hardware vs. Software Flow Control
If feasible, it's best to use "hardware" flow control that uses two dedicated "modem control" wires to send the "stop" and "start" signals. Hardware flow control at the serial port works like this: The two pins, RTS (Request to send) and CTS (Clear to send) are used. When the computer is ready to receive date it asserts RTS by putting a positive voltage on the RTS pin (meaning "Request To Send to me"). When the computer is not able to receive any more bytes, it negates RTS by putting a negative voltage on the pin saying: "stop sending to me". The RTS pin is connected by the serial cable to another pin on the modem, printer, terminal, etc. This other pin's only function is to receive this signal.
For the case of a modem, this "other" pin will be the modem's RTS pin. But for a printer, another PC, or a non-modem device, it's usually a CTS pin so a "crossover" or "null modem" cable is required. This cable connects the CTS pin at one end with the RTS pin at the other end (two wires since each end of the cable has a CTS pin). For a modem, a straight-thru cable is used.
For the opposite direction of flow a similar scheme is used. For a modem, the CTS pin is used to send the flow control signal to the CTS pin on the PC. For a non-modem, the RTS pin sends the signal. Thus modems and non-modems have the roles of their RTS and CTS pins interchanged. Some non-modems such as dumb terminals may use other pins for flow control such as the DTR pin instead of RTS.
Software flow control uses the main receive and transmit data wires to send the start and stop signals. It uses the ASCII control characters DC1 (start) and DC3 (stop) for this purpose. They are just inserted into the regular stream of data. Software flow control is not only slower in reacting but also does not allow the sending of binary data unless special precautions are taken. Since binary data will likely contain DC1 and DC3 characters, special means must be taken to distinguish between a DC3 that means a flow control stop and a DC3 that is part of the binary code. Likewise for DC1. To get software flow control to work for binary data requires both modem (hardware) and software support.
Symptoms of No Flow Control
Understanding flow-control theory can be of practical use. For example I used my modem to access the Internet and it seemed to work fine. But after a few months, I tried to send out long files from my PC and a huge amount of retries and errors resulted but it finally succeeded. Receiving in the other direction (from my ISP to me) worked fine. The problem turned out to be a modem with flow control disabled. My modem's buffer was overflowing (overrunning) on long outgoing files since no "stop" signal was ever sent to my computer to halt sending to the modem. There was no problem in the direction from the modem to my computer since the capacity (say 115.2 kbps) of the serial port was always higher than the flow from the telephone line. But it was a problem in the other direction where the PC would send at 115.2 kbps and the modem couldn't handle this high flow resulting in overruns of the modem's buffer. The fix was to enable flow control by putting into the modem's init string an enable-flow-control command.
Modem-to-Modem Flow Control
This is the flow control of the data sent over the telephone lines between two modems. It works as long as error correction is enabled. Actually, even without error correction it's possible to enable software flow control between modems but it may interfere with sending binary data so it's not often used.
4.10 Data Flow Path; Buffers
It's been mentioned that there are 3 buffers for each direction of flow (3 pairs altogether): 16-byte FIFO buffers (in the UART), a pair of larger buffers inside a device connected to the serial port (such as a modem), and a pair of buffers (say 8k) in main memory. When an application program sends bytes to the serial port they first get stashed in the transmit serial port buffer in main memory. The other member of this pair consists of a receive buffer for the opposite direction of byte-flow. Here's an example diagram for the case of browsing the Internet with a browser. Transmit data flow is left to right while receive flow is right to left. There is a separate buffer for each direction of flow.
application 8k-byte 16-byte 1k-byte tele- BROWSER ------- MEMORY -------- FIFO --------- MODEM -------- phone program buffer buffer buffer line
For the transmit case, the serial device driver takes out say 15 bytes from this transmit buffer (in main memory), one byte at a time and puts them into the 16-byte transmit buffer in the serial UART for transmission. Once in that transmit buffer, there is no way to stop them from being transmitted. They are then transmitted to the modem or (other device connected to the serial port) which also has a fair sized (say 1k) buffer. When the device driver (on orders from flow control sent from the modem) stops the flow of outgoing bytes from the computer, what it actually stops is the flow of outgoing bytes from the large transmit buffer in main memory. Even after this has happened and the flow to the modem has stopped, an application program may keep sending bytes to the 8k transmit buffer until it becomes fill. At the same time, the bytes stored in the FIFO and continue to be sent out. Bytes stored in the modem will continue to be sent out the phone line unless the modem has gotten a modem-to-modem flow control stop from the modem at the other end of the phone line.
When the memory buffer gets fill, the application program can't send any more bytes to it (a "write" statement in a C-program blocks) and the application program temporarily stops running and waits until some buffer space becomes available. Thus a flow control "stop" is ultimately able to stop the program that is sending the bytes. Even though this program stops, the computer does not necessarily stop computing since it may switch to running other processes while it's waiting at a flow control stop.
The above was a little oversimplified in three ways. First, some UARTs can do automatic hardware flow control which can stop the transmission out of the FIFO buffers if needed (not yet supported by Linux). Second, while an application process is waiting to write to the transmit buffer, it could possibly perform other tasks. Third, the serial driver (located between the memory buffer and the FIFO) has it's own small buffer (in main memory) used to process characters.
4.11 Modem Commands
Commands to the modem are sent to it from the communication software over the same conductor as used to send data. The commands are short ASCII strings. Examples are "AT&K3" for enabling hardware flow control (RTS/CTS) between your computer and modem; and "ATDT5393401 for Dialing the number 5393401. Note all commands are prefaced by "AT". Some commands such as enabling flow control help configure the modem. Other commands such as dialing a number actually do something. There are about a hundred or so different possible commands. When your communication software starts running, it first sends an "init" string of commands to the modem to configure it. All commands are sent on the ordinary data line before the modem dials (or receives a call).
Once the modem is connected to another modem (on-line mode), everything that is sent from your computer to your modem goes directly to the other modem and is not interpreted by the modem as a command. There is a way to "escape" from this mode of operation and go back to command mode where everything sent to the modem will be interpreted as a command. The computer just sends "+++" with a specified time spacing before and after it. If this time spacing is correct, the modem reverts to command mode. Another way to do this is by a signal on a certain modem control line.
There are a number of lists of modem commands on the Internet. The section Web Sites has links to a couple of such web-sites. Different models and brands of modems do not use exactly the same set of such commands. So what works for one modem might not work for another. Some common commands (not guaranteed to work on all modems) are listed in this HOWTO in the section Modem Configuration
4.12 Serial Driver Module
The device driver for the serial port is the software that
operates the serial port. It is now provided as a serial module.
From kernel 2.2 on, this module will normally get loaded automatically
if it's needed. In earlier kernels, you had to have kerneld
running in order to do auto-load modules on demand. Otherwise the
serial module needed to be explicitly listed in /etc/modules. Before
modules became popular with Linux, the serial driver was usually built
into the kernel (and sometimes still is). If it's built-in don't let
the serial module load or else you will have two serial drivers
running at the same time. With 2 drivers there are all sorts of
errors including a possible "I/O error" when attempting to open a
serial port. Use "lsmod" to see if the module is loaded.
When the serial module is loaded it displays a message on the screen
about the existing serial ports (often showing a wrong IRQ). But once
the module is used by setserial to tell the device driver the
(hopefully) correct IRQ then you should see a second display similar
to the first but with the correct IRQ, etc. See
"Serial Module" in the Serial-HOWTO.
See
What is Setserial for more info on
setserial.
Next Previous Contents