7. Debugging tips and programming information
7.1 Submitting useful problem reports
The best way to submit reports is to use the online pcmcia-cs forums or the bug tracker at SourceForge. That way, other people can see current problems (and fixes or workarounds, if available). Here are some things that should be included in all bug reports:
- Your system brand and model.
- All PCMCIA card(s) you are using.
- Your Linux kernel version (i.e., ``
uname -rv''), and PCMCIA driver version (i.e., ``cardctl -V''). - Output of 'lspci -v'
- Any changes you have made to the startup files in
/etc/pcmcia, or to the PCMCIA startup script. - All PCMCIA-related messages in your system log file. That includes startup messages, and messages generated when your cards are configured.
All the PCMCIA modules and the cardmgr daemon send status
messages to the system log. These will usually end up somewhere like
/var/log/messages or /var/log/daemon.log. These
files should be the first place to look when tracking down a problem.
When submitting a bug report, always include the relevant contents of
these files. If you are having trouble finding your system messages,
check /etc/syslog.conf to see how different classes of
messages are handled.
Before submitting a bug report, please check to make sure that you are using an up-to-date copy of the driver package. While it is somewhat gratifying to read bug reports for things I've already fixed, it isn't a particularly constructive use of my time.
If you do not have web access, bug reports can be sent to me at
dahinds@users.sourceforge.net. However, I prefer that
bug reports be posted to the pcmcia-cs SourceForge site, so that they
can be seen by others.
7.2 Interpreting kernel trap reports
If your problem involves a kernel fault, the register dump from the
fault is only useful if you can translate the fault address, EIP, to
something meaningful. Recent versions of klogd attempt to
translate fault addresses based on the current kernel symbol map, but
this may not work if the fault is in a module, or if the problem is
severe enough that klogd cannot finish writing the fault
information to the system log.
If a fault is in the main kernel, the fault address can be looked up
in the System.map file. This may be installed in
/System.map or /boot/System.map. If a fault is in a
module, the nm command gives the same information, however, the
fault address has to be adjusted based on the module's load address.
Let's say that you have the following kernel fault:
Unable to handle kernel NULL pointer dereference
current->tss.cr3 = 014c9000, %cr3 = 014c9000
*pde = 00000000
Oops: 0002
CPU: 0
EIP: 0010:[<c2026081>]
EFLAGS: 00010282
The fault address is 0xc2026081. Looking at System.map, we
see that this is past the end of the kernel, i.e., is in a kernel
module. To determine which module, check the output of
``ksyms -m | sort'':
Address Symbol Defined by
c200d000 (35k) [pcmcia_core]
c200d10c register_ss_entry [pcmcia_core]
c200d230 unregister_ss_entry [pcmcia_core]
...
c2026000 (9k) [3c574_cs]
c202a000 (4k) [serial_cs]
So, 0xc2026081 is in the 3c574_cs module, and is at an offset of
0x0081 from the start of the module. We cannot look up this offset in
3c574_cs.o yet: when the kernel loads a module, it inserts a
header at the module load address, so the real start of the module is
offset from the address shown in ksyms. The size of the header
varies with kernel version: to find out the size for your kernel,
check a module that exports symbols (like pcmcia_core above), and
compare a symbol address with nm output for that symbol. In this
example, register_ss_entry is loaded at an offset of 0xc200d10c -
0xc200d000 = 0x010c, while ``nm pcmcia_core.o'' shows the offset
as 0x00c0, so the header size is 0x010c - 0x00c0 = 0x004c bytes.
Back to 3c574_cs, our fault offset is 0x0081, and subtracting the
0x004c header, the real module offset is 0x0035. Now looking at
``nm 3c574_cs.o | sort'', we see:
0000002c d if_names
0000002c t tc574_attach
00000040 d mii_preamble_required
00000041 d dev_info
So, the fault is located in tc574_attach().
In this example, the fault did not cause a total system lockup, so
ksyms could be executed after the fault happened. In other
cases, you may have to infer the module load addresses indirectly.
The same sequence of events will normally load modules in the same
order and at the same addresses. If a fault happens when a certain
card is inserted, get the ksyms output before inserting the card,
or with a different card inserted. You can also manually load the
card's driver modules with insmod and run ksyms before
inserting the card.
For more background, see ``man insmod'', ``man ksyms'', and
``man klogd''. In the kernel source tree,
Documentation/oops-tracing.txt is also relevant. Here are a
few more kernel debugging hints:
- Depending on the fault, it may also be useful to translate addresses in the ``Call Trace'', using the same procedure as for the main fault address.
- To diagnose a silent lock-up, try to provoke the problem with X disabled, since kernel messages sent to the text console will not be visible under X.
- If you kill
klogd, most kernel messages will be echoed directly on the text console, which may be helpful if the problem preventsklogdfrom writing to the system log. - To cause all kernel messages to be sent to the console, for 2.2
and later kernels, if
/proc/sys/kernel/printkexists, do:echo 8 > /proc/sys/kernel/printk
- The key combination <RightAlt><ScrLk> prints a register dump on the text console. This may work even if the system is otherwise completely unresponsive, and the EIP address can be interpreted as for a kernel fault.
- For 2.2 and later kernels configured with
CONFIG_MAGIC_SYSRQenabled, various emergency functions are available via special <Alt><SysRq> key combinations, documented inDocumentation/sysrq.txtin the kernel source tree.
7.3 Low level PCMCIA debugging aids
The PCMCIA modules contain a lot of conditionally-compiled debugging
code. Most of this code is under control of the PCMCIA_DEBUG
preprocessor define. If this is undefined, debugging code will
not be compiled. If set to 0, the code is compiled but inactive.
Larger numbers specify increasing levels of verbosity. Each module
built with PCMCIA_DEBUG defined will have an integer parameter,
pc_debug, that controls the verbosity of its output. This
can be adjusted when the module is loaded, so output can be controlled
on a per-module basis without recompiling.
Your default configuration for syslogd may discard kernel
debugging messages. To ensure that they are recorded, edit
/etc/syslog.conf to ensure that ``kern.debug'' messages
are recorded somewhere. See ``man syslog.conf'' for details.
There are a few register-level debugging tools in the
debug_tools/ subdirectory of the PCMCIA distribution. The
dump_tcic and dump_i365 utilities generate register dumps
for ISA-to-PCMCIA controllers. In 3.1.15 and later releases,
dump_i365 is replaced by dump_exca, which is similar but
also works for PCI-to-CardBus bridges. Also new in 3.1.15 for CardBus
bridges is the dump_cardbus tool, which interprets the
CardBus-specific registers. These are all most useful if you have
access to a datasheet for the corresponding controller chip. The
dump_cis utility (dump_tuples in pre-3.0.2 distributions)
lists the contents of a card's CIS (Card Information Structure), and
decodes most of the important bits. And the dump_cisreg utility
displays a card's local configuration registers.
The memory_cs memory card driver is also sometimes useful for
debugging problems with 16-bit PC Cards. It can be bound to any card,
and does not interfere with other drivers. It can be used to directly
access any card's attribute memory or common memory. Similarly for
CardBus cards, the memory_cb driver can be bound to any 32-bit
card, to give direct access to that card's address spaces. See the
man pages for more information.
7.4 /proc/bus/pccard
On 2.2 and later kernels, the PCMCIA package will create a tree
of status information under /proc/bus/pccard.
Much of the information can only be interpreted using the data sheets
for the PCMCIA host controller. Its contents may depend on how the
drivers were configured, but may include all or some of the following:
/proc/bus/pccard/{irq,ioport,memory}If present, these files contain resource allocation information to supplement the normal kernel resource tables. Recent versions of the PCMCIA system may obtain additional resource information from the Plug and Play BIOS if configured to do so.
/proc/bus/pccard/driversIn recent releases, this lists all currently loaded PCMCIA client drivers. Unlike
/proc/modules, it also lists drivers that may be statically linked into the kernel./proc/bus/pccard/*/infoFor each socket, describes that socket's host controller and its capabilities.
/proc/bus/pccard/*/excaThis contains a dump of a controller's ``ExCA'' Intel i82365sl-compatible register set.
/proc/bus/pccard/*/{pci,cardbus}For CardBus bridges, a dump of the bridge's PCI configuration space, and a dump of the bridge's CardBus configuration registers.
7.5 Writing Card Services drivers for new cards
The Linux PCMCIA Programmer's Guide is the best documentation for the
client driver interface. The latest version is always available from
projects.sourceforge.net in /pub/pcmcia-cs/doc, or on
the web at
http://pcmcia-cs.sourceforge.net.
For devices that are close relatives of normal ISA devices, you will probably be able to use parts of existing Linux drivers. In some cases, the biggest stumbling block will be modifying an existing driver so that it can handle adding and removing devices after boot time. Of the current drivers, the memory card driver is the only ``self-contained'' driver that does not depend on other parts of the Linux kernel to do most of the dirty work.
In many cases, the largest barrier to supporting a new card type is obtaining technical information from the manufacturer. It may be difficult to figure out who to ask, or to explain exactly what information is needed. However, with a few exceptions, it is very difficult if not impossible to implement a driver for a card without technical information from the manufacturer.
I have written a dummy driver with lots of comments that explains a
lot of how a driver communicates with Card Services; you will find
this in the PCMCIA source distribution in clients/dummy_cs.c.
7.6 Guidelines for PCMCIA client driver authors
I have decided that it is not really feasible for me to distribute all PCMCIA client drivers as part of the PCMCIA package. Each new driver makes the main package incrementally harder to maintain, and including a driver inevitably transfers some of the maintenance work from the driver author to me. Instead, I will decide on a case by case basis whether or not to include contributed drivers, based on user demand as well as maintainability. For drivers not included in the core package, I suggest that driver authors adopt the following scheme for packaging their drivers for distribution.
Driver files should be arranged in the same directory scheme used in
the PCMCIA source distribution, so that the driver can be unpacked on
top of a complete PCMCIA source tree. A driver should include source
files (in ./modules/), a man page (in ./man/), and
configuration files (in ./etc/). The top level directory
should also include a README file.
The top-level directory should include a makefile, set up so
that ``make -f ... all'' and ``make -f ... install'' compile the
driver and install all appropriate files. If this makefile is given
an extension of .mk, then it will automatically be invoked by the
top-level Makefile for the all and install targets.
Here is an example of how such a makefile could be constructed:
# Sample Makefile for contributed client driver
FILES = sample_cs.mk README.sample_cs \
modules/sample_cs.c modules/sample_cs.h \
etc/sample.conf etc/sample etc/sample.opts \
man/sample_cs.4
all:
$(MAKE) -C modules MODULES=sample_cs.o
install:
$(MAKE) -C modules install-modules MODULES=sample_cs.o
$(MAKE) -C etc install-clients CLIENTS=sample
$(MAKE) -C man install-man4 MAN4=sample_cs.4
dist:
tar czvf sample_cs.tar.gz $(FILES)
This makefile uses install targets defined in 2.9.10 and later
versions of the PCMCIA package. This makefile also includes a
``dist'' target for the convenience of the driver author. You would
probably want to add a version number to the final package filename
(for example, sample_cs-1.5.tar.gz). A complete distribution
could look like:
sample_cs.mk
README.sample_cs
modules/sample_cs.c
modules/sample_cs.h
etc/sample.conf
etc/sample
etc/sample.opts
man/sample_cs.4
With this arrangement, when the contributed driver is unpacked, it becomes essentially part of the PCMCIA source tree. It can make use of the PCMCIA header files, as well as the machinery for checking the user's system configuration, and automatic dependency checking, just like a ``normal'' client driver.
In this example, etc/sample and etc/sample.opts
would be the new driver's configuration scripts (if needed), and
etc/sample.conf would contain any additions to the PCMCIA
card configuration file. Starting with the 3.1.6 release,
cardmgr will automatically process any *.conf files
installed in /etc/pcmcia, so installation of contributed
drivers should no longer require hand editing configuration files.
I will accept client drivers prepared according to this specification
and place them in the /pub/pcmcia-cs/contrib directory on
projects.sourceforge.net. The README in this directory will
describe how to unpack a contributed driver.
The client driver interface has not changed much over time, and has almost always preserved backwards compatibility. A client driver will not normally need to be updated for minor revisions in the main package. I will try to notify authors of contributed drivers of changes that require updates to their drivers.
7.7 Guidelines for Linux distribution maintainers
If your distribution has system configuration tools that you would
like to be PCMCIA-aware, please use the *.opts files in
/etc/pcmcia for your ``hooks.'' These files will not be
modified if a user compiles and installs a new release of the PCMCIA
package. If you modify the main configuration scripts, then a fresh
install will silently overwrite your custom scripts and break the
connection with your configuration tools. Contact me if you are not
sure how to write an appropriate option script, or if you need
additional capabilities.
It is helpful for users (and for me) if you can document how your distribution deviates from the PCMCIA package as described in this document. In particular, please document changes to the startup script and configuration scripts. If you send me the appropriate information, I will include it in the Notes about specific Linux distributions.
When building PCMCIA for distribution, consider including contributed drivers that are not part of the main PCMCIA package. For reasons of maintainability, I am trying to limit the core package size, by only adding new drivers if I think they are of particularly broad interest. Other drivers will be distributed separately, as described in the previous section. The split between integral and separate drivers is somewhat arbitrary and partly historical, and should not imply a difference in quality.
Next Previous Contents