NetWorker – an advanced web server with a microcrontroller

An Internet connection would be a valuable addition to many projects, but often designers are put off by the complexities involved. The ‘NetWorker’, which consists of a small printed circuit board, a free software library and a ready-to-use microcontroller-based web server, solves these problems and allows beginners to add Internet connectivity to their projects. More experienced users will benefit from features such as SPI communications, power over Ethernet (PoE) and more.

There are three key elements in connecting a device to an Ethernet network: the Ethernet connection hardware itself, the software library (called the ‘stack’) responsible for handling the various protocol layers, and finally the top-level firmware functions. For example, to allow access to a device from anywhere in the world, it is possible to use a small web server running in a microcontroller.

Compact ready-made modules are available on the market to cover the first two of these elements. They allow you to connect a microcontroller to a network without having to get involved in the gritty details of its protocols. However, programming against the libraries provided can be a daunting task for the beginner; and more experienced users often find it necessary to ‘work around’ the firmware provided with the module to enhance or modify it.

The solution provided here hides the complexity of the network protocols from the application programmer, and is simple to extend and very versatile. The circuit is based around a Microchip PIC18-family microcontroller which includes a built-in Ethernet transceiver. The author has added a few new functions to the free C software library available from the manufacturer that implements the TCP/IP stack. The final element of the project is a web server running on the microcontroller which can communicate with other hardware via the I/O pins of the device. Using this design it is possible to assemble a small system in just a few minutes, capable of being controlled at will over the Internet. Beyond that, the module can be used as a ‘network modem’ for another microcontroller. Extra advanced features such as PoE round out the project. Printed circuit boards and ready-programmed microcontrollers are available from Elektor, as well as the complete readymade module. And of course the accompanying software is also available as a free download from the Elektor website. Now let’s see what makes the module tick.

NetWorker - an advanced web server with a microcrontroller

The circuit

Figure 1 (below) shows the circuit of the module. At the heart of the unit is a Microchip PIC18F67J60. This device is one member of a range of microcontrollers that include a 10BASE-T Ethernet transceiver. Most of the circuitry required is provided in the device and only a few external components (including a connector!) are needed to connect to a network.

The Ethernet PICs require a 2.5 V core voltage supply, which can be derived from a higher voltage (3.3 V is recommended) using the voltage regulator included in the PIC. Pin ENVREG determines whether the voltage regulator is enabled. A 10 μF capacitor (C112) provides smoothing and capacitors C101 to C106 positioned around the microcontroller decouple the supplies. A 25 MHz crystal (Q101) provides an external clock for the microcontroller, and the Ethernet transceiver clock is also derived from this source.

The project includes a bootloader which allows new firmware to be uploaded over the Ethernet interface. Alternatively the device can be programmed via the solder pads MCLR, VCC, GND, PGD and PGC which form the standard Microchip in-circuit serial programming (ICSP) interface. This interface is needed for debugging software problems or if the module gets into a state where it no longer responds to being rebooted. A dedicated programmer (such as the ICD2) is required for debugging and programming over this interface. The reset circuit comprises R102, R103 and C107. R102 is a pull-up resistor, and R103 and C107 filter the reset signal to prevent spurious resets. C107 can interfere with the operation of the ICSP interface, and so should be removed if it is to be used. The EEPROM stores configuration data. It also includes a 48-bit ID code, called the MAC address, required for Ethernet communication. The proprietary Microchip UNI/O bus protocol allows the EEPROM to be accessed over a single wire.

Ethernet interface

The connection to the Ethernet network comprises four signals, TPOUT+, TPOUT–, TPIN+ and TPIN–. These signals are passed via the Ethernet transformer TR1, which galvanically isolates the main part of the circuit from the network. On the far side of the transformer is an RJ45 socket J1, into which a standard Ethernet cable can be plugged. A special feature of the transformer we have chosen is its compatibility with the IEEE 802.3af power over Ethernet (PoE) standard. Under this standard a 48 V supply is provided on the network and a powered device (PD) can draw current from this supply. There are two alternative arrangements possible:

1. using the spare conductor pairs (on J1 these are pins 4 and 5 and pins 7 and 8); or
2. phantom supply over the signal pairs (pins 1 and 2 and pins 3 and 6).

A PoE-standard device must support both possibilities. To take advantage of the phantom supply a special transformer is needed with centre taps to the windings on the network side.

Implementing both power options means that a total of four wires have to be brought to headers JP1 and JP2, which form the userside interface to the module. A suitable PoE regulator circuit can be connected to these headers to derive a supply for the module from the 48 V DC provided over the network (see below).

Resistors R107 to R110 provide impedance matching, and C108, C109 and FE101 help reduce the effect of interference spikes on the network connections. R104 is a bias resistor that provides the Ethernet transceiver circuit with a known reference current (hence its bizarre value). This in turn determines the signal amplitude on TPOUT+ and TPOUT–.

There are two LEDs mounted inside the Ethernet socket. LED1 lights when the link is up (in other words, when there is a connection between the module and at least one other network device), and LED2 indicates network activity. The LEDs are driven from the PIC via series current-limiting resistors R105 and R106. User-side interface

The user-side interface consists of two 10-way 0.1 inch pitch headers. As well as carrying the PoE connections and the module’s power supply, the headers are also directly connected to certain pins on the microcontroller. On the author’s printed circuit board design (see Figure 2 right) the two headers are mounted 0.1 inch apart, which allows the module to be mounted easily on ordinary prototyping board. Pins VCC and GND form the power supply to the module. The supply voltage should be between 3.1 V and 3.6 V. Pins VA1, VA2, VB1 and VB2 allow the module to be used in conjunction with a power over Ethernet regulator, sometimes called a ‘PD module’, such as the Silver Telecom Ag9000 series [2]. Figure 3 (below) shows an example circuit using the Ag9033, which provides a regulated 3.3 V output from the PoE DC supply.

 

  • Technique

Technology

 

NetWorker – an advanced web server with a microcrontroller

An Internet connection would be a valuable addition to many projects, but often designers are put off by the complexities involved. The ‘NetWorker’, which consists of a small printed circuit board, a free software library and a ready-to-use microcontroller-based web server, solves these problems and allows beginners to add Internet connectivity to their projects. More experienced users will benefit from features such as SPI communications, power over Ethernet (PoE) and more.

There are three key elements in connecting a device to an Ethernet network: the Ethernet connection hardware itself, the software library (called the ‘stack’) responsible for handling the various protocol layers, and finally the top-level firmware functions. For example, to allow access to a device from anywhere in the world, it is possible to use a small web server running in a microcontroller.

Compact ready-made modules are available on the market to cover the first two of these elements. They allow you to connect a microcontroller to a network without having to get involved in the gritty details of its protocols. However, programming against the libraries provided can be a daunting task for the beginner; and more experienced users often find it necessary to ‘work around’ the firmware provided with the module to enhance or modify it.

The solution provided here hides the complexity of the network protocols from the application programmer, and is simple to extend and very versatile. The circuit is based around a Microchip PIC18-family microcontroller which includes a built-in Ethernet transceiver. The author has added a few new functions to the free C software library available from the manufacturer that implements the TCP/IP stack. The final element of the project is a web server running on the microcontroller which can communicate with other hardware via the I/O pins of the device. Using this design it is possible to assemble a small system in just a few minutes, capable of being controlled at will over the Internet. Beyond that, the module can be used as a ‘network modem’ for another microcontroller. Extra advanced features such as PoE round out the project. Printed circuit boards and ready-programmed microcontrollers are available from Elektor, as well as the complete readymade module. And of course the accompanying software is also available as a free download from the Elektor website. Now let’s see what makes the module tick.

The circuit

Figure 1 (below) shows the circuit of the module. At the heart of the unit is a Microchip PIC18F67J60. This device is one member of a range of microcontrollers that include a 10BASE-T Ethernet transceiver. Most of the circuitry required is provided in the device and only a few external components (including a connector!) are needed to connect to a network.

The Ethernet PICs require a 2.5 V core voltage supply, which can be derived from a higher voltage (3.3 V is recommended) using the voltage regulator included in the PIC. Pin ENVREG determines whether the voltage regulator is enabled. A 10 μF capacitor (C112) provides smoothing and capacitors C101 to C106 positioned around the microcontroller decouple the supplies. A 25 MHz crystal (Q101) provides an external clock for the microcontroller, and the Ethernet transceiver clock is also derived from this source.

The project includes a bootloader which allows new firmware to be uploaded over the Ethernet interface. Alternatively the device can be programmed via the solder pads MCLR, VCC, GND, PGD and PGC which form the standard Microchip in-circuit serial programming (ICSP) interface. This interface is needed for debugging software problems or if the module gets into a state where it no longer responds to being rebooted. A dedicated programmer (such as the ICD2) is required for debugging and programming over this interface. The reset circuit comprises R102, R103 and C107. R102 is a pull-up resistor, and R103 and C107 filter the reset signal to prevent spurious resets. C107 can interfere with the operation of the ICSP interface, and so should be removed if it is to be used. The EEPROM stores configuration data. It also includes a 48-bit ID code, called the MAC address, required for Ethernet communication. The proprietary Microchip UNI/O bus protocol allows the EEPROM to be accessed over a single wire.

Ethernet interface

The connection to the Ethernet network comprises four signals, TPOUT+, TPOUT–, TPIN+ and TPIN–. These signals are passed via the Ethernet transformer TR1, which galvanically isolates the main part of the circuit from the network. On the far side of the transformer is an RJ45 socket J1, into which a standard Ethernet cable can be plugged. A special feature of the transformer we have chosen is its compatibility with the IEEE 802.3af power over Ethernet (PoE) standard. Under this standard a 48 V supply is provided on the network and a powered device (PD) can draw current from this supply. There are two alternative arrangements possible:

1. using the spare conductor pairs (on J1 these are pins 4 and 5 and pins 7 and 8); or
2. phantom supply over the signal pairs (pins 1 and 2 and pins 3 and 6).

A PoE-standard device must support both possibilities. To take advantage of the phantom supply a special transformer is needed with centre taps to the windings on the network side.

Implementing both power options means that a total of four wires have to be brought to headers JP1 and JP2, which form the userside interface to the module. A suitable PoE regulator circuit can be connected to these headers to derive a supply for the module from the 48 V DC provided over the network (see below).

Resistors R107 to R110 provide impedance matching, and C108, C109 and FE101 help reduce the effect of interference spikes on the network connections. R104 is a bias resistor that provides the Ethernet transceiver circuit with a known reference current (hence its bizarre value). This in turn determines the signal amplitude on TPOUT+ and TPOUT–.

There are two LEDs mounted inside the Ethernet socket. LED1 lights when the link is up (in other words, when there is a connection between the module and at least one other network device), and LED2 indicates network activity. The LEDs are driven from the PIC via series current-limiting resistors R105 and R106. User-side interface

The user-side interface consists of two 10-way 0.1 inch pitch headers. As well as carrying the PoE connections and the module’s power supply, the headers are also directly connected to certain pins on the microcontroller. On the author’s printed circuit board design (see Figure 2 right) the two headers are mounted 0.1 inch apart, which allows the module to be mounted easily on ordinary prototyping board. Pins VCC and GND form the power supply to the module. The supply voltage should be between 3.1 V and 3.6 V. Pins VA1, VA2, VB1 and VB2 allow the module to be used in conjunction with a power over Ethernet regulator, sometimes called a ‘PD module’, such as the Silver Telecom Ag9000 series [2]. Figure 3 (below) shows an example circuit using the Ag9033, which provides a regulated 3.3 V output from the PoE DC supply.

The active-low MCLR pin allows the module to be reset. It is also pulled low briefly when power is applied to the module. The other pins on the user-side interface are for use either to control external circuits or to sense their state, or for more general communications (by UART or SPI). Pins GPIO0 to GPIO2 can be configured as digital inputs or outputs; those configured as inputs can be used to generate an interrupt to the microcontroller. Used as an output, each pin can sink or source up to 25 mA. Pin GPIO3 is also configurable as a digital input or output, but can only sink or source up to 8 mA. A future software version may allow a PWM signal to be generated on this pin. GPIO4 and GPIO5 (which can be configured as inputs or outputs) can sink or source up to 2 mA. A future software version may allow these pins to be used as analogue inputs with 10 bit conversion resolution.

Communications

Pins TX and RX are connected to a UART that can support speeds of up to 115200 baud. Alternatively, these pins can be configured as ordinary digital inputs or outputs; as outputs, they can each sink or source up to 25 mA.

An SPI interface is available on the pins with labels starting ‘SPI’. The interface can operate in master mode or in slave mode, and the function of the pins depends on the mode selected.

1. In master mode SPI_INT is not used, SPI_CLK is the clock output, SPI_MOSI the data output, SPI_MISO the data input. The active-low SPI_CS output is used to enable the connected slave device.
2. In slave mode SPI_INT is used to indicate when an event has occurred in the module that needs the attention of the SPI bus master. SPI_CLK is the clock input, SPI_MOSI the data output, and SPI_MISO the data input. The active-low SPI_CS input is used to enable reception of a bus message by the module.

Again, these pins can alternatively be configured as digital inputs or outputs. As an output, SPI_CS can sink or source up to 2 mA, while the other pins can each sink or source up to 25 mA. A future software version may allow SPI_ MISO and SPI_CLK to be used as an I2C interface. In this case SPI_MISO would be the SDA (data) signal and SPI_MOSI the SCL (clock) signal.

Software library

A ‘stack’ is a collection of software implementations of protocols and drivers, usually arranged in a hierarchy of layers. At the bottom end of the stack are the hardware drivers which are responsible for getting data bits transferred onto the network wires. At the top end of the stack is a simple interface for data exchange.

In theory, the layered protocol model makes it relatively easy to make modifications to one layer (such as the hardware driver) without affecting the others. In practice however, when implemented on a microcontroller, the hardware and the stack are very closely tied together, and lots of tricks need to be used to keep memory usage low. Nevertheless, TCP/IP stack implementations are available for a wide range of microcontroller families, often for free and direct from the manufacturer.

The author’s first tentative steps towards building a TCP/IP stack for a PIC microcontroller were in assembler. Over time, however, as the code grew, maintenance became harder and harder, and an alternative solution was needed.

Microchip offers a TCP/IP stack for its PIC18, PIC24, dsPIC and PIC32 microcontroller families written in ANSI C. It is free and can be extended and modified as long as it is only used in Microchip devices. It contains a rich collection of hardware drivers, lowlevel protocol implementations (ARP, IP, TCP and so on) and a couple of important application-level protocol implementations such as DHCP and HTTP. The simple MPFS file system is also included to allow storage of web page source data. Example applications, a bootloader, tools and comprehensive documentation of all stack functions complete the package.

The author evaluated and extended this protocol stack for the project in the light of his experience designing the ‘eWicht’ handheld model railway controller [3]. For example, the DHCP module was extended to include an AutoIP mechanism (see glossary). An mDNS server was incorporated so that the module could autonomously make its presence known on a network (see below).

A feature was added to check values entered on web forms for type and range errors, and the MPFS tool provided by the manufacturer was extended to use a Huffman code to obtain a saving in memory footprint of up to 50 %.

Application software

The module has an integrated bootloader that allows a new version of its firmware to be uploaded over the Ethernet connection for installation at any time: suitable hex files are available on the project’s web pages [1]

NetWorker - an advanced web server with a microcrontroller Schematic

Bootloader

The bootloader used is the one included in the TCP/IP software package provided by Microchip for its PIC18F97J60-series devices. This includes a footprint-optimised implementation of the TFTP server and ARP protocols (see glossary), as well as a hex file parser and flash programming routines: everything needed to program new code into the chip.

There are two ways to access the bootloader. If the microcontroller has not been programmed with an application hex file, or if there has been an error programming the hex file, the bootloader is automatically invoked. This will always be the case with a ‘virgin’ microcontroller. Alternatively, if an application hex file has been successfully programmed, the bootloader will be active for just the first two seconds or so after power is applied. The bootloader’s IP address is fixed at 192.168.97.60, and its MAC address is 00-04-A3-00-00- 00. A connected PC must have an address on the same network (such as 192.168.97.61) or it will not be possible to upload a new file. The microcontroller is then programmed by simply sending a hex file using TFTP.

If you wish to upload a new hex file while the device is in operation, you simply need to start a TFTP upload to its current IP address. The bootloader will be invoked automatically and the programming operation will start. (For this to work the running application must make use of the ‘Reboot’ module, which is part of the Microchip TCP/IP stack.) TFTP clients are available for almost all operating systems. A typical invocation of the client on a Windows system looks like this:

tftp 192.168.97.60 put “file.hex”

After a short wait a message resembling the following should appear:

Transfer successful: 203557 bytes in 8 seconds, 25444 bytes/s

The device will then automatically reboot into the new version of the firmware.

The author developed the application software for the module using the MPLAB IDE. It is written entirely in ANSI C and can be compiled using the free version of the Microchip C18 compiler.

The application consists of three modules, called Main, Web and Appl. The bootloader jumps directly to Main, where the TCP/IP stack, the EEPROM driver and the Appl module are initialised. Then Appl and the TCP/IP stack receive processing time alternately in an infinite loop.


About The Author

Ibrar Ayyub

I am an experienced technical writer holding a Master's degree in computer science from BZU Multan, Pakistan University. With a background spanning various industries, particularly in home automation and engineering, I have honed my skills in crafting clear and concise content. Proficient in leveraging infographics and diagrams, I strive to simplify complex concepts for readers. My strength lies in thorough research and presenting information in a structured and logical format.

Follow Us:
LinkedinTwitter

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top