New HOWTO: Modem-HOWTO - page 5

Table of Contents

  4.  Serial Port and Modem Basics

  You don't have to understand the basics to use and install a modem.
  But understanding it may help to determine what is wrong if you run
  into problems.  After reading this section, if you want to understand
  it even better you may want to see ``How Modems Work'' in this
  document (not yet complete).  More details on the serial port
  (including much of this section) will be found in Serial-HOWTO.


  4.1.  Modem Converts Digital to Analog (and conversely)

  Most all telephone main lines are digital already but the lines
  leading to your house (or business) are usually analog which means
  that they were designed to transmit a voltage wave which is an exact
  replica of the sound wave coming out of your mouth.  Such a voltage
  wave is called "analog".  If viewed on an oscilloscope it looks like a
  sine wave of varying frequency and amplitude.  A digital signal is
  like a square wave.  For example 3 v (volts) might be a 1-bit and 0 v
  could be a 0-bit.  For most serial ports (used by external modems) +12
  v is a 0-bit and -12 v is a 1-bit (some are + or - 5 v).

  To send data from your computer over the phone line, the modem takes
  the digital signal from your computer and converts it to "analog".  It
  does this by both creating an analog sine wave and then "MODulating"
  it.  Since the result still represents digital data, it could also be
  called a digital signal instead of analog.  But it looks something
  like an analog signal and almost everyone calls it analog.  At the
  other end of the phone line another modem "DEModulates" this signal
  and the pure digital signal is recovered.  Put together the "mod" and
  "dem" parts of the two words above and you get "modem" (if you drop
  one of the two d's).  A "modem" is thus a MODulator-DEModulator.  Just
  what modulation is may be found in the section ``Modulation Details''.



  4.2.  What is a Serial Port ?

  4.2.1.  Intro to Serial

  The serial port is an I/O (Input/Output) device.  Since modems have a
  serial port between them and the computer, it's necessary to
  understand the serial port as well as the modem.

  Most PC's have one or two serial ports.  Each has a 9-pin connector
  (sometimes 25-pin) on the back of the computer.  Computer programs can
  send data (bytes) to the transmit pin (output) and receive bytes from
  the receive pin (input).  The other pins are for control purposes and
  ground.

  The serial port is much more than just a connector.  It converts the
  data from parallel to serial and changes the electrical representation
  of the data.  Inside the computer, data bits flow in parallel (using
  many wires at the same time).  Serial flow is a stream of bits over a
  single wire (such as on the transmit or receive pin of the serial
  connector).  For the serial port to create such a flow, it must
  convert data from parallel (inside the computer) to serial on the
  transmit pin (and conversely).

  Most of the electronics of the serial port is found in a computer chip
  (or a part of a chip) known as a UART.  For more details on UARTs see
  the section "What are UARTS" in the Serial-HOWTO.

  But you may want to finish this section first so that you will
  hopefully understand how the UART fits into the overall scheme of
  things.


  4.2.2.  Pins and Wires

  Old PC's used 25 pin connectors but only about 9 pins were actually
  used so today most connectors are only 9-pin.  Each of the 9 pins
  usually connects to a wire.  Besides the two wires used for
  transmitting and receiving data, another pin (wire) is signal ground.
  The voltage on any wire is measured with respect to this ground.  Thus
  the minimum number of wires to use for 2-way transmission of data is
  3.  Except that it has been known to work with no signal ground wire
  but with degraded performance and sometimes with errors.

  There are still more wires which are for control purposes (signalling)
  only and not for sending bytes.  All of these signals could have been
  shared on a single wire, but instead, there is a separate dedicated
  wire for every type of signal.  Some (or all) of these control wires
  are called "modem control lines".  Modem control wires are either in
  the asserted state (on) of +12 volts or in the negated state (off) of
  -12 volts.  One of these wires is to signal the computer to stop
  sending bytes out the serial port cable.  Conversely, another wire
  signals the device attached to the serial port to stop sending bytes
  to the computer.  If the attached device is a modem, other wires may
  tell the modem to hang up the telephone line or tell the computer that
  a connection has been made or that the telephone line is ringing
  (someone is attempting to call in).  See the Serial-HOWTO: Pinout and
  Signals for more details.



  4.2.3.  Internal Modem Contains Serial Port

  For an internal modem there is no 9-pin connector but the behavior is
  almost exactly as if the above mentioned cable wires existed.  Instead
  of a a 12 volt signal in a wire giving the state of a modem control
  line, the internal modem may just use a status bit in its own memory
  (a register) to determine the state of this non-existent "wire".  The
  internal modem's serial port looks just like a real serial port to the
  computer.  It even includes the speed limits that one may set at
  ordinary serial ports such as 115200 bits/sec.  Unfortunately for
  Linux, many internal modems today don't work exactly this way but
  instead use software (running on the CPU) to do much of the modem's
  work.  Unfortunately, such software is often only available for the MS
  Windows OS (it hasn't been ported to Linux).  Thus you can't use most
  of these modems with Linux See ``Software-based Modems (winmodems)''.



  4.3.  IO Address & IRQ

  Since the computer needs to communicate with each serial port, the
  operating system must know that each serial port exists and where it
  is (its I/O address).  It also needs to know which wire (IRQ number)
  the serial port must use to request service from the computer's CPU.
  It requests service by sending an interrupt on this wire.  Thus every
  serial port device must store in its non-volatile memory both its I/O
  address and its Interrupt ReQuest number: IRQ.  See ``Interrupts''.
  For the PCI bus it doesn't work exactly this way since the PCI bus has
  its own system of interrupts.  But since the PCI-aware BIOS sets up
  chips to map these PCI interrupts to IRQs, it seemingly behaves just
  as described above except that sharing of interrupts is allowed (2 or
  more devices may use the same IRQ number).

  I/O addresses are not the same as memory addresses.  When an I/O
  addresses is put onto the computer's address bus, another wire is
  energized.  This both tells main memory to ignore the address and
  tells all devices which have I/O addresses (such as the serial port)
  to listen to the address to see if it matches the device's.  If the
  address matches, then the I/O device reads the data on the data bus.


  4.4.  Names: ttyS0, ttyS1, etc.

  The serial ports are named ttyS0, ttyS1, etc. (and usually correspond
  respectively to COM1, COM2, etc. in DOS/Windows).  The /dev directory
  has a special file for each port.  Type "ls /dev/ttyS*" to see them.
  Just because there may be (for example) a ttyS3 file, doesn't
  necessarily mean that there exists a physical serial port there.

  Which one of these names (ttyS0, ttyS1, etc.) refers to which physical
  serial port is determined as follows.  The serial driver (software)
  maintains a table showing which I/O address corresponds to which ttyS.
  This mapping of names (such as ttyS1) to I/O addresses (and IRQ's) may
  be both set and viewed by the "setserial" command.  See ``What is
  Setserial''.  This does not set the I/O address and IRQ in the
  hardware itself (which is set by jumpers or by plug-and-play
  software).  Thus what physical port corresponds to say ttyS1 depends
  both on what the serial driver thinks (per setserial) and what is set
  in the hardware.  If a mistake has been made, the physical port may
  not correspond to any name (such as ttyS2) and thus it can't be used.
  See ``Serial Port Devices /dev/ttyS2, etc.'' for more details>



  4.5.  Interrupts


  Bytes come in over the phone line to the modem, are converted from
  analog to digital by the modem and passed along to the serial port on
  their way to their destination inside your computer.  When the serial
  port receives a number of bytes (may be set to 1, 4, 8, or 14) into
  its FIFO buffer, it signals the CPU to fetch them by sending an
  electrical signal known as an interrupt on a certain wire normally
  used only by that port.  Thus the FIFO waits for a number of bytes and
  then issues an interrupt.

  However, this interrupt will also be sent if there is an unexpected
  delay while waiting for the next byte to arrive (known as a timeout).
  Thus if the bytes are being received slowly (such as someone typing on
  a terminal keyboard) there may be an interrupt issued for every byte
  received.  For some UART chips the rule is like this: If 4 bytes in a
  row could have been received, but none of these 4 show up, then the
  port gives up waiting for more bytes and issues an interrupt to fetch
  the bytes currently in the FIFO.  Of course, if the FIFO is empty, no
  interrupt will be issued.

  Each interrupt conductor (inside the computer) has a number (IRQ) and
  the serial port must know which conductor to use to signal on.  For
  example, ttyS0 normally uses IRQ number 4 known as IRQ4 (or IRQ 4).  A
  list of them and more will be found in "man setserial" (search for
  "Configuring Serial Ports").  Interrupts are issued whenever the
  serial port needs to get the CPU's attention.  It's important to do
  this in a timely manner since the buffer inside the serial port can
  hold only 16 (1 in old serial ports) incoming bytes.  If the CPU fails
  to remove such received bytes promptly, then there will not be any
  space left for any more incoming bytes and the small buffer may
  overflow (overrun) resulting in a loss of data bytes.

  For an external modem, there is no way (such as flow control) to stop
  the flow rapidly enough to prevent this.  For an internal modem the
  16-byte FIFO buffer is on the same card and a good modem will not
  write to it if it's full.  Thus a good internal modem will not overrun
  the 16-byte buffers but it may need to use ``Modem-to-Modem Flow
  Control'' to prevent the modem itself from being overrun.  This is one
  advantage of an internal modem over an external.

  Interrupts are also issued when the serial port has just sent out all
  16 of its bytes from its small transmit buffer out the external cable.
  It then has space for 16 more outgoing bytes.  The interrupt is to
  notify the CPU of that fact so that it may put more bytes in the small
  transmit buffer to be transmitted.  Also, when a modem control line
  changes state an interrupt is issued.

  The buffers mentioned above are all hardware buffers.  The serial port
  also has large buffers in main memory.  This will be explained later

  Interrupts convey a lot of information but only indirectly.  The
  interrupt itself just tells a chip called the interrupt controller
  that a certain serial port needs attention.  The interrupt controller
  then signals the CPU.  The CPU runs a special program to service the
  serial port.  That program is called an interrupt service routine
  (part of the serial driver software).  It tries to find out what has
  happened at the serial port and then deals with the problem such a
  transferring bytes from (or to) the serial port's hardware buffer.
  This program can easily find out what has happened since the serial
  port has registers at IO addresses known to the the serial driver
  software.  These registers contain status information about the serial
  port.  The software reads these registers and by inspecting the
  contents, finds out what has happened and takes appropriate action.

  4.6.  Data Compression (by the Modem)

  Before continuing with the basics of the serial port, one needs to
  understand about something done by the modem: data compression.  In
  some cases this task is actually done by software run on the
  computer's CPU but unfortunately at present, such software only works
  for MS Windows.  The discussion here will be for the case where the
  modem itself does the compression since this is what must happen in
  order for the modem to work under Linux.

  In order to send data faster over the phone line, one may compress
  (encode it) using a custom encoding scheme which itself depends on the
  data.  The encoded data is smaller than the original (less bytes) and
  can be sent over the Internet in less time.  This process is called
  "data compression".

  If you download files from the Internet, they are likely already
  compressed and it is not feasible for the modem to try to compress
  them further.  Your modem may sense that what is passing thru has
  already been compressed and refrain from trying a compress it any
  more.  If you are receiving data which has been compressed by the
  other modem, your modem will decompress it and create many more bytes
  than were sent over the phone line.  Thus the flow of data from your
  modem into your computer will be higher than the flow over the phone
  line to you.  The ratio of this flow is called the compression ratio.
  Compression ratios as high as 4 are possible, but not very likely.


  4.7.  Error Correction

  Similar to data compression, modems may be set to do error correction.
  While there is some overhead cost involved which slows down the
  byte/sec flow rate, the fact that error correction strips off start
  and stop bits actually increases the data byte/sec flow rate.

  For the serial port's interface with the external world, each 8-bit
  byte has 2 extra bits added to it: a start-bit and a stop-bit.
  Without error correction, these extra start and stop bits usually go
  right thru the modem and out over the phone lines.  But when error
  correction is enabled, these extra bits are stripped off and the 8-bit
  bytes are put into packets.  This is more efficient and results in
  higher byte/sec flow in spite of the fact that there are a few more
  bytes added for packet headers and error correction purposes.


  4.8.  Data Flow (Speeds)

  Data (bytes representing letters, pictures, etc.) flows from your
  computer to your modem and then out on the telephone line (and
  conversely).  Flow rates (such as 56k (56000) bits/sec) are
  (incorrectly) called "speed".  But almost everyone says "speed"
  instead of "flow rate".  If there were no data compression the flow
  rate from the computer to the modem would be about the same as the
  flow rate over the telephone line.

  Actually there are two different speeds to consider at your end of the
  phone line:


  �  The speed on the phone line itself (DCE speed) modem-to-modem

  �  The speed from your computer's serial port to your modem (DTE
     speed)

  When you dial out and connect to another modem on the other end of the
  phone line, your modem often sends you a message like "CONNECT 28800"
  or "CONNECT 115200".  What do these mean?  Well, its either the DCE
  speed or the DTE speed. If it's higher than the advertised modem speed
  it must be the DTE modem-to-computer speed.  This is the case for the
  115200 speed shown above.  The 28800 must be a DCE (modem-to-modem)
  speed since the serial port has no such speed.  One may configure the
  modem to report either speed.  Some modems report both speeds and
  report the modem-to-modem speed as (for example): CARRIER 28800.

  If you have an internal modem you would not expect that there would be
  any speed limit on the DTE speed from your modem to your computer
  since you modem is inside your computer and is almost part of your
  computer.  But there is since the modem contains a dedicated serial
  port within it.

  It's important to understand that the average speed is often less than
  the specified speed, especially on the short DTE computer-to-modem
  line.  Waits (or idle time) result in a lower average speed.  These
  waits may include long waits of perhaps a second due to ``Flow
  Control''.  At the other extreme there may be very short waits (idle
  time) of several micro-seconds separating the end of one byte and the
  start of the next byte.  In addition, modems will fallback to lower
  speeds if the telephone line conditions are less than pristine.

  For a discussion of what DTE speed is best to use see section ``What
  Speed Should I Use''.



  4.9.  Flow Control

  Flow control means the ability to slow done the flow of bytes in a
  wire.  For serial ports this means the ability to stop and then
  restart the flow without any loss of bytes.  Flow control is needed
  for modems to allow a jump in instantaneous flow rates.


  4.9.1.  Example of Flow Control

  For example, consider the case where you connect a 36.6k external
  modem via a short cable to your serial port.  The modem sends and
  receives bytes over the phone line at  36.6k bits per second (bps).
  Assume it's not doing any data compression or error correction.  You
  have set the serial port speed to 115,200 bits/sec (bps), and you are
  sending data from your computer to the phone line.  Then the flow from
  the your computer to your modem over the short cable is at 115.2k bps.
  However the flow from your modem out the phone line is only 33.6k bps.
  Since a faster flow (115.2k) is going into your modem than is coming
  out of it, the modem is storing the excess flow (115.2k -33.6k = 81.6k
  bps) in one of its buffers.  This buffer would soon overrun (run out
  of free storage space) unless the high 115.2k flow is stopped.

  But now flow control comes to the rescue.  When the modem's buffer is
  almost full, the modem sends a stop signal to the serial port.  The
  serial port passes on the stop signal on to the device driver and the
  115.2k bps flow is halted.  Then the modem continues to send out data
  at 33.6k bps drawing on the data it previous accumulated in its
  buffer.  Since nothing is coming into the buffer, the level of bytes
  in it starts to drop.  When almost no bytes are left in the buffer,
  the modem sends a start signal to the serial port and the 115.2k flow
  from the computer to the modem resumes.  In effect, flow control
  creates an average flow rate in the short cable (in this case 33.6k)
  which is significantly less than the "on" flow rate of 115.2k bps.
  This is "start-stop" flow control.


  In the above simple example it was assumed that the modem did no data
  compression.  This would be true when the modem is sending a file
  which is already compressed and can't be compressed further.  Now
  let's consider the opposite extreme where the modem is compressing the
  data with a high compression ratio.  In such a case the modem might
  need an input flow rate of say 115.2k bps to provide an output (to the
  phone line) of 33.6k bps (compressed data).  The compression ratio is
  3.43 (115.2/33.6) which is much higher than average.  In this case the
  modem is able to compress and the 115.2 bps PC-to-modem flow and send
  the same data out on the phone line at 33.6bps.  There's no need for
  flow control here.  But such a high compression ratio rarely happens
  so that most of the time flow control is needed to slow down the flow
  on the 115.2 bps PC-to-modem cable.  The flow is stopped and started
  so that the average flow is usually well under the "on" flow of 115.2
  bps.

  In the above example the modem was an external modem.  But the same
  situation exists (as of late 2000) for most internal modems.  There is
  still a speed limit on the PC-to-modem speed even though this flow
  doesn't take place over an external cable.  This makes the internal
  modems compatible with the external modems.

  In the above example of flow control the flow was from the computer to
  a modem.  But there is also flow control which is used for the
  opposite direction of flow: from a modem (or other device) to a
  computer.  Each direction of flow involve 3 buffers: 1. in the modem
  2. in the UART chip (called FIFOs) 3. in main memory managed by the
  serial driver.  Flow control protects certain buffers from
  overflowing.  The small UART FIFO buffers are not protected in this
  way but rely instead on a fast response to the interrupts they issue.
  FIFO stand for "First In, First Out" which is the way it handles
  bytes.  All the 3 buffers use the FIFO rule but only one of them also
  uses it as a name.  This is the essence of flow control but there are
  still some more details.

  You don't often need flow control in the direction from the modem to a
  PC.  For complex example of a case where it's needed see "Complex Flow
  Control Example" in the Serial-HOWTO.  But if you don't have a high
  enough speed set between the modem and the computer (serial port
  speed) then you do need to slow down the flow coming into the modem
  from the telephone line.  To do this your modem must tell the other
  modem to stop sending.  See ``Modem-to-Modem Flow Control''.



  4.9.2.  Hardware vs. Software Flow Control

  If feasible it's best to use "hardware" flow control that uses two
  dedicated "modem control" wires to send the "stop" and "start"
  signals.  Modern modems almost always use hardware flow control
  between the modem and the serial port.

  Software flow control uses the main receive and transmit wires to send
  the start and stop signals.  It uses the ASCII control characters DC1
  (start) and DC3 (stop) for this purpose.  They are just inserted into
  the regular stream of data.  Software flow control is not only slower
  in reacting but also does not allow the sending of binary data unless
  special precautions are taken.  Since binary data will likely contain
  DC1 and DC3, special means must be taken to distinguish between a DC3
  that means a flow control stop and a DC3 that is part of the binary
  code.  Likewise for DC1.  To get software flow control to work for
  binary data requires both modem (hardware) and software support



  4.9.3.  Symptoms of No Flow Control

  Understanding flow-control theory can be of practical use.  For
  example I used my modem to access the Internet and it seemed to work
  fine.  But after a few months I tried to send long files from my PC to
  an ISP and a huge amount of retries and errors resulted (but
  eventually Kermit could send a long file after many retries).
  Receiving in the other direction (from my ISP to me) worked fine.  The
  problem turned out to be a modem with broken flow control.  My modem's
  buffer was overflowing (overrunning) on long outgoing files since no
  "stop" signal was ever sent to the computer to halt sending to the
  modem.  There was no problem in the direction from the modem to my
  computer since the capacity (say 115.2k) was always higher than the
  flow over the telephone line.  The fix was to enable flow control by
  putting into the init string an enable-flow-control command for the
  modem.  (It should have been enabled by default but something was
  wrong).


  4.9.4.  Modem-to-Modem Flow Control

  This is the flow control of the data sent over the telephone lines
  between two modems.  Practically speaking, it only exists when you
  have error correction enabled.  Actually, even without error
  correction it's possible to enable software flow control between
  modems but it may interfere with sending binary data so it's not often
  used.


  4.10.  Data Flow Path; Buffers

  Much has been explained about this including flow control, a pair of
  16-byte FIFO buffers (in the UART), and a pair of larger buffers
  inside a device connected to the serial port (such as a modem.  But
  there is still another pair of buffers.  These are large buffers
  (perhaps 8k) in main memory also known as serial port buffers.  When
  an application program sends bytes to the serial port (and modem) they
  first get stashed in the the transmit serial port buffer in main
  memory.  The pair consists of both this transmit buffer and a receive
  buffer for the opposite direction of byte-flow.  Here's an example
  diagram for the case of browsing the Internet with a browser.
  Transmit data flow is left to right while receive flow is right to
  left.


  application     8k-byte         16-byte        1k-byte        tele-
  BROWSER ------- MEMORY -------- UART --------- MODEM -------- phone
  program         buffer          buffer         buffer         line



  The serial device driver takes out say 16 bytes from this transmit
  buffer, one byte at a time and puts them into the 16-byte transmit
  buffer in the serial UART for transmission.  Once in that transmit
  buffer, there is no way to stop them from being transmitted.  They are
  then transmitted to the modem which also has a fair sized (say 1k)
  buffer.  When the device driver (on orders from flow control) stops
  the flow of outgoing bytes from the computer, what it actually stops
  is the flow of outgoing bytes from the large transmit buffer in main
  memory.  Even after this has happened and the flow to the modem has
  stopped, an application program may keep sending bytes to the 8k
  transmit buffer until it becomes fill.

  When it gets fill, the application program can't send any more bytes
  to it (a "write" statement in a C_program blocks) and the application
  program temporarily stops running and waits until some buffer space
  becomes available.  Thus a flow control "stop" is ultimately able to
  stop the program that is sending the bytes.  Even though this program
  stops, the computer does not necessarily stop computing.  It may
  switch to running other processes while it's waiting at a flow control
  stop.  The above was a little oversimplified since there is another
  alternative of having the application program itself do something else
  while it is waiting to "write".


  4.11.  Modem Commands

  Commands to the modem are sent to it from the communication software
  over the same conductor as used to send data.  The commands are short
  ASCII strings.  Examples are "AT&K3" for enabling hardware flow
  control (RTS/CTS) between your computer and modem; and "ATDT5393401
  for Dialing the number 5393401.  Note all commands are prefaced by
  "AT".   Some commands such as enabling flow control help configure the
  modem.  Other commands such as dialing a number actually do something.
  There are about a hundred or so different possible commands.  When
  your communication software starts running, it first sends an "init"
  string of commands to the modem to configure it.  All commands are
  sent on the ordinary data line before the modem dials (or receives a
  call).

  Once the modem is connected to another modem (on-line mode),
  everything that is sent from your computer to your modem goes directly
  to the other modem and is not interpreted by the modem as a command.
  There is a way to "escape" from this mode of operation and go back to
  command mode where everything sent to the modem will be interpreted as
  a command.  The computer just sends "+++" with a specified time
  spacing before and after it.  If this time spacing is correct, the
  modem reverts to command mode.  Another way to do this is by a signal
  on a certain modem control line.

  There are a number of lists of modem commands on the Internet.  The
  section ``Web Sites'' has links to a couple of such web-sites.
  Different models and brands of modems do not use exactly the same set
  of such commands.  So what works for one modem might not work for
  another.  Some common command (not guaranteed to work on all modems)
  are listed in this HOWTO in the section ``Modem Configuration''



  4.12.  Serial Driver Module

  The device driver for the serial port is the software that operates
  the serial port.  It is now provided as a serial module.  From kernel
  2.2 on, this module will normally get loaded automatically if it's
  needed.  In earlier kernels, you had to have kerneld running in order
  to do auto-load modules on demand.  Otherwise the serial module needed
  to be explicitly listed in /etc/modules.  Before modules became
  popular with Linux, the serial driver was usually built into the
  kernel (and sometimes still is).  If it's built-in don't let the
  serial module load or else you will have two serial drivers running at
  the same time.  With 2 drivers there are all sorts of errors including
  a possible "I/O error" when attempting to open a serial port.  Use
  "lsmod" to see if the module is loaded.

  When the serial module is loaded it displays a message on the screen
  about the existing serial ports (often showing a wrong IRQ).  But once
  the module is used by setserial to tell the device driver the
  (hopefully) correct IRQ then you should see a second display similar
  to the first but with the correct IRQ, etc.  See "Serial Module" in
  the Serial-HOWTO.  See ``What is Setserial'' for more info on
  setserial.