How-To: Web server on a business card (Part 2)

This mini web server is slightly smaller than a business card. There are a lot of tiny one-board servers out there, but this is probably the smallest you can etch and solder at home. Unlike many embedded web servers, files are stored on a PC-readable SD card, not in a difficult-to-write EEPROM. Read on for the web server design, or catch up on PIC 24F basics in the previous article: Web server on a business card (part 1).

Concept overview

The goal of this project is to build a web server on a business card that serves web pages and files from a FAT formatted SD card. The server is based on a PIC 24F that connects to a TCP/IP network using the ENC28J60 ethernet MAC/PHY. Network layers and low-level services, such as DNS and DHCP, are handled by the Microchip TCP/IP stack. A FAT 12/16/32 formatted SD card contains web pages and files. A very simple HTTP server ties everything together by handling page requests on port 80, searching the SD card for requested, and serving them with the correct content type.

Hardware

(full size schematic .png)

Microcontroller (Microchip PIC 24FJ64GA002)

The brain of the server is a 16-bit PIC 24FJ64GA002 (IC1), a 28pin microcontroller available in several hobbyist friendly packages. Check out our PIC 24F introduction for more about working with this chip.

PIC 24Fs operate between 2 and 3.8volts, which is perfect because the ethernet chip (IC2) and SD card both run at 3.3volts. This chip has 8K of RAM, plenty for the TCP/IP stack and a few K for working with a full FAT file system. The 24FJ64 has two SPI modules, so the SD card and ethernet IC each get a dedicated data bus.

The PIC processor core operates at 2.5volts, and requires a 10uF capacitor (C2) for the on-chip voltage regulator. The datasheet specifies a tantalum capacitor, but we used a low-ESR electrolytic in a prototype without incident. Every power pin needs a 0.1uF decoupling capacitor (C4,5).

The internal 8MHz oscillator provides a 32MHz clock source with the 4x PLL multiplier enabled. We’re also using an external 32.768KHz crystal (Q1) with 2 x 27pF capacitors (C17,18) to enable the real time clock calendar.

Programming connections are brought to a header (SV1). We chose to use programming pin pair three (PGx3). The master clear and reset (MCLR) function is enabled with a 2K resistor (R1) from V+ to the MCLR pin. Optionally, add a button (S1) from MCLR to ground for a manual reset switch.

Ethernet connection (ENC28J60)

An ENC28J60 (IC2) handles the network physical connection (PHY) and MAC layer. The ENC28J60 needs a number of support parts beyond the typical 0.1uF decoupling capacitors (C6,7,9,10). A 25MHz crystal (Q2) and 2 x 27pf capacitors (C15,16) provide a clock signal. The internal core voltage regulator requires a 10uF tantalum capacitor (C1), but an electrolytic capacitor also worked fine. Two LEDs (LED1,2) with 330ohm resistors (R2,3) display link and data status.

A bias resistor (R12) is required; the value will depend on the ENC28J60 version you’re using. Current chips should be B5 (PDF) or B7 (PDF), and require a 2.32K 1% resistor.

The PHY I/O portion specifies 4 x 49.9ohm 1% resistors (R8-11), and a ferrite bead (L1).

The most difficult-to-find part for the ENC28J60 is the correct RJ-45 jack with integrated magnetics (RJ1). We used a J1006F21 PulseJack from Pulse Engineering. Be sure to check the pin configuration and connections if you use a different jack, they will probably be different than ours. A Cadsoft Eagle part library for the JP1006F21 is included in the project archive. This was a $4 part, but it’s gone up to $7. If you know of other jacks that work we’ll add them here.

microSD card

We used a microSD/transflash card in this design because SD cards waste a lot of board space under the holder. microSD cards are smaller versions of SD cards with the same data interface, and most come with an adapter for use in standard SD card readers. The card needs a holder (SD1) and a 0.1uF decoupling capacitor (C8).

If you want to use a full-size SD card, take a look at our version one prototype in the project archive. We used Alps SD card holder #SCDA1A0901. Unfortunately, this part is has been discontinued and we’ve yet to find a suitable replacement. Don’t try #SCDA5A0201, that’s for sure. If you have a favorite, we’ll add it here. Sparkfun has one, and a matching Cadsoft Eagle part library.

Power supply

An adjustable LM317 voltage regulator (IC3) is set to 3.3volts using a 390ohm (R6) and 240ohm (R7) resistor. We considered several 3.3volt regulators, but nothing was cheaper than a LM317 and two resistors. There’s a 0.1uF decoupling capacitor (C13,14) and a 10uF capacitor (C3,19) on both sides to help support the power hungry Ethernet transceiver. The LM317 will output 3.3volts from an input of 5 to 20volts+, but it gets really hot with greater than 9volts supply. The specified input capacitor is only rated 16volts, so consider an upgrade if you plan to use a supply greater than about 9volts.

For the first time ever, we incorporated a power jack (J1) into a design. A jack with a 2.1mm diameter internal pin seems to be the most common DC connector. We used a cheap through-hole DC power jack, like SparkFun #PRT-00119 or Mouser #163-7620-E. It mates with a plug like Mouser #1710-0721.

Circuit board

The PCB (full size placement .png) was designed in Cadsoft Eagle 5.0. Freeware versions are available for all major platforms. Renderings were done with Eagle3D, beta version. Schematic and board files are included in the project archive (ZIP).

We designed the project with large SOIC chips and 0805 surface mount (SMD) parts, but haters can rest assured that chips are available in a through-hole package. We prefer to use SMD parts because the resulting circuit boards are smaller, cheaper, and faster to produce. 0805 parts are dirt cheap, and easy to solder with a normal iron. Don’t expect this project to work on a breadboard, there’s probably too much capacitance for this circuit.

We took full advantage of the PIC’s programmable pin placement to get the simplest trace routings possible. Just four jumper wires are needed on an otherwise single-sided board.

The traces are large and clean, DIY toner transfer boards should be easy. We made our PCB using an inkjet printer transparency mask over an UV sensitive circuit board.

In addition to the final design, the project archive contains our v1 prototype design. The prototype uses a full size SD card (SCDA1A0901) and all electrolytic 10uF capacitors. We also put the RJ45 Ethernet jack on a daughterboard to better accommodate different pinouts.

Partslist

# Count Part Size Mouser#
IC1 1 PIC 24FJ64GA002-SO SOIC-28 579-PIC24FJ64GA002SO
IC2 1 ENC28J60 SOIC-28 579-ENC28J60-I/SO
IC3 1 LM317 voltage regulator D2Pack 511-LM317D2T-TR
C1-3 3 10uF tantalum capacitor A case 74-293D106X96R3A2TE3
C4-14 11 0.1uF capacitor 0805 80-C0805C104M5R
C15-18 4 27pF capacitor 0805 140-CC501N270J-RC
C19 1 10uF capacitor 0805 647-UWF1C100MCL1GB
R1 1 2K0ohm resistor 0805 292-2.0K-RC
R2-6 5 390ohm resistor 0805 71-CRCW0805-390
R7 1 240ohm resistor 0805 71-CRCW0805-240
R8-11 4 49.9ohm 1% resistor 0805 71-CRCW0805-49.9-E3
R12 1 2K32ohm 1% resistor 0805 71-CRCW0805-2.32K-E3
L1 1 Ferrite bead 0805 81-BLM21BB600SN1D
LED1-4 4 LED 0805 645-598-8110-107F
Q1 1 32.768KHz crystal SMD 695-CM200S-327KF-U
Q2 1 25MHz crystal HC49 815-ABLS-25-B2
RJ1 1 Pulse J1006F21 - 673-J1006F21
S1 1 Tactile switch DTS-6 101-0164-EV
SV1,2 11 .1″ male pin header - 571-41033290
J1 1 Power jack - 163-7620-E
SD1 1 microSD card holder - SparkFun: PRT-00127

Firmware

Three firmware examples are included in the project archive [zip]. The examples compile with Microchip’s demonstration C30 compiler. Learn more about working with the PIC 24F in our previous article: Web server on a business card (part 1). MPLAB isn’t great about project portability, you may need to locate all the project files again if your path doesn’t match the ‘c:wsbc’ format that we used.

FAT12/16/32 disk library

Our first step was to get the FAT library reading from a SD card. FAT 12/16/32 are simple disk storage formats that work with PCs, MACs, digital cameras, music players, and other electronics. Here’s our favorite FAT tutorial/teardown (PDF).

Microchip’s FAT 12/16/32 library gives us simple functions for working with SD cards. The included demo application creates some files and directories to demonstrate each function. Here’s how we configured it to work on our custom hardware, you can find these changes by searching for the tag ‘HACKADAY’ in the code:

  • HardwareProfile.h assigns actual PIC hardware to generic references in the code library. For the SD card this is an SPI interface, and pins for chip select and card detect. First, we deleted all the unused hardware profiles to make the code more manageable. Next, we configured the FAT library to communicate with the SD card using an SPI module (line 132). Finally, we defined the SPI pin assignments (line 152). Pin setup is shown in the table below.
    Pin Port
    Chip select B0
    SD card detect A2
    SPI clock B2
    SPI MOSI B1
    SPI MISO B3
  • Demonstration.c. On line 48 we set a custom oscillator fuse configuration, as described in our PIC 24F introduction. This is also the logical place to configure pin assignments with peripheral pin select (line 63).
  • FSConfig.h. This file enables various components of file system library, affecting the amount of memory and program space used. A read-only library is very small, a full write configuration is bigger. We didn’t have to make any changes for the demonstration, but this is an important file to note.

At first, the library failed to recognize our SD card. It only supports disks with a master boot record (MBR). Windows XP formats SD cards as a DOS disk: a single partition with no MBR. To verify this, open a Windows-formatted disk with a utility like HxD and inspect sector 0 of the physical disk. Byte 446 should be the location of the first MBR partition entry, but instead it’s the NTLDR executable code.

To format the disk in the ‘correct’ FAT format, use a digital camera’s format function or a utility like Panasonic’s SD card formatter. We also considered using a different FAT library that reads DOS disks, like DOSFS, or adding similar features to the Microchip firmware.

TCP/IP stack

Microchip’s free TCP/IP stack performs the convoluted configuration and networking functions needed to run a web server. You can read all about the stack in various application notes and documentation. Wikipedia is our favorite TCP/IP learning resource; we wrote our first TCP/IP stack using only Wikipedia.

Microchip’s TCP/IP stack used to be messy and confusing. Now it’s just confusing. The last few versions of have improved considerably in code clarity and structure. Here’s what we did to to configure the base TCP/IP stack example for our hardware, you can find these changes by searching for the tag ‘HACKADAY’ in the code:

  • HardwareProfile.h assigns actual PIC hardware resources to generic references in the code library. We added our custom oscillator configuration (line 68), and configured the server status LED to use the LED attached to PORTB7 (line 83).  We defined the SPI interface to the ENC28J60 as follows (line 116):
    Pin Port
    Reset B8
    Chip select B9
    SPI clock B10
    SPI MOSI B11
    SPI MISO B12
    Wake on lan B13
    Interrupt B14
  • MainDemo.c. We eliminated a bunch of unused code, and added the peripheral pin select configuration code to the InitializeBoard() function (line 332).
  • TCPIPConfig.h defines the TCPIP stack components included in a compile. We’ve enabled DNS, DHCP, the IP announcer, and the ping server (line 56):
#define STACK_USE_DNS            // Domain Name Service Client
#define STACK_USE_DHCP_CLIENT    // Get DNS automagically
#define STACK_USE_ANNOUNCE       // Microchip Ethernet Device Discoverer
#define STACK_USE_ICMP_SERVER    // Enable the PING server

After loading this firmware, we’re ready to connect the server to a network for the first time. During initialization, the TCP/IP stack negotiates with the network router for an IP address using DHCP. We need to know this address to communicate with the device. If the device had a screen we could display the IP address, but instead we use the MCHPDetect.exe utility from Microchip.

When the TCP/IP stack finishes initializing, it broadcasts an announcement packet to port 30303 of all locally connected computers. MCHPDetect extracts the IP address from these packets. A new announce packet is sent on every PIC reset.

It’s also possible to read the IP address directly from memory with a debugger. The address is stored in the AppConfig.MyIPAddr variable, the .byte form follows the standard x.x.x.x IP notation.

Once we have the IP address, we can ping the server and test its responsiveness.

If ping shows high latency or malformed packets, you can use Wireshark to inspect network traffic at the byte level. Unless you’re in Germany, because it might be criminal.

Building the custom HTTP server

The custom web server looks for requested files on the SD card, and sends them with the correct content type. We used the Microchip HTTP example server v1 (HTTP.c) as a base for our FAT file server (FATHTTP.c).

Microchip’s HTTP server used a simple file system called MPFS to index web pages on an EEPROM chip. We replaced calls to MPFS functions with calls to functions in the FAT library (see the HTTPProcess and Sendfile functions in FATHTTP.c). Our changes demonstrate the concept as simply as possible, without adding confusing pointers and other handy C obfuscations. The code leaves a ton of room for improvements, have at it. File writes are disabled in the default compilation, but there’s enough program space to enable them if you want to write to the SD card (see FSConfig.h).

It’s necessary to registered our custom FATHTTP server with the rest of the TCP/IP stack. We did a search and replace for the original HTTP server components, and added calls to our new FATHTTP server as needed. That turned out to be these places:

  • TCPIPConfig.h. First we inserted some definitions that enable the FATHTTP server (line 70), and added a TCP socket for the FATHTTP server (line 248).
  • TCPIP.h. Next, we added FATHTTP to the list of services that require the TCP/IP stack (line 170) and then included the necessary headers (line 351).
  • StackTSK.c. We added the FATHTTP server initialization (line 138) and processing (line 340) functions to the list of TCP/IP stack tasks.
  • Helpers.c. We also needed to include a few helper functions for working with URLs (line 259).

At long last, it’s time to put some files on an SD card and test this thing. Make sure your files follow the 8.3 file name format. The project archive contains a sample website with a test image and zip file.

After grabbing the server’s IP address with MCHPDetect, we pointed a browser at it. The IP address entered alone will redirect the browser to index.htm, whether or not it exists. Web pages and images stored on the SD card display in the browser, but unknown binary types trigger a download prompt.

Taking it further

We see a lot of potential projects using this tiny web platform.

  • Add hooks in the FATHTTP.c source for special URLs that trigger events or configure pins.
  • Build a remotely accessible data logger. Use the extra pins to read sensors and log data to the SD card. Logs are retrievable from a web browser, or directly from the FAT readable SD card.
  • Get remote access to an ancient serial terminal or BBS, optionally log the console output. Use two external pins as a serial port, and forward commands from the Internet using Microchip’s Telnet server and Ethernet-to-serial bridge examples.
  • Your suggestions?

Next time, we’ll use the mini server to make an Internet connected, electronic indoor graffiti wall. This will be an interactive project where everyone can contribute graffiti and animations on-line.

Schematic, board, and firmware files are included in the project archive (ZIP). Use the freeware version of Cadsoft Eagle to view the schematic and PCB. The firmware is written in C, and compiled with the Microchip demonstration C30 compiler.

Comments

  1. I am having problem with the first link. It gives a 404 error?

  2. Jordan S says:

    There is one really important fact that should be noted about the ENC28J60 Ethernet Controller Chip. It DOES NOT have a pre-programmed MAC address. That means if you plan on selling your product that uses it you need to buy a MAC address or a block of addresses. They cost like $1500 for a block of a couple million from IEEE but Microchip sells them burned into an EEPROM for about $0.40 a piece and then you can just read in the address with SPI or I2C when your device boots…

  3. bhavin says:

    I am just wondering if MRF24WB0MA can be a direct replacement for ENC28J60 here and wifi enable this business card. After all, business cards should not be wired

  4. mike says:

    maybe its the years of being a drone, but what exactly could i do with one of these.

    i honestly have no ideas beyond a links page with a few tools

  5. Garrick says:

    I see you’ve swapped the MPFS with FAT file system. According to the documentation, the MPFS filesystem formats html file data into a structure with separate indexes to dynamic variables in the html file. I don’t know what the advantage is, but I need to upload files directly without converting them to this format. Is it correct that the rest of the stack does not need access the to internal structures of the MPFS? Thanks.

  6. esteban says:

    Anybody have information how to use telnet server to interact with a telnet client to set a pin?

  7. Basil Watson says:

    Hi I tried the webserver out with both the WSBC Webserver and the WSBC SD Demo. These work ok. With the Graffiti server I can MHCPdetect and Ping server but I cannot get web page .. not found. I have comnented out lines for ‘web page” and am using SD card. What lines need to to be in or out? IF I uncomment the line FSInit in main.c, then I can’t detect server at all and led4 does not work. Appears to be something with SD card not being read. However it works with the other two apps.

    Thanks, for a great demo project hope to do more when I learn more .

  8. Ian says:

    There is no web server on the graffiti board, it is a client for the graffiti wall only.

  9. Basil Watson says:

    Thanks, for info will test more. Get panel working.

  10. Ry says:

    How easy would it be to swap out the SD card with a flash memory chip (SPI)? Anyone know if this is possile.

  11. Ian says:

    Sure, the microchip stack supports that. You could even have both with just one extra chip select signal.

  12. Michelle says:

    Looks nice, I was planning a similar project and was wondering what kind of latency you were experiencing with that particular camera, and if it was a problem with controlling the car. Thanks for the article!

  13. Tom Hargrave says:

    Looks like the prokect files are NLA from the original server. Does anyone have a copy?

    Thanks in advance, Tom

  14. Ashkan says:

    Hi!
    http://blog.mahalo.com/hackaday/howto/wsbcv3.zip i think this link is removed
    please upload again
    thanks

  15. EZequiel says:

    Hey please, does anybody have the wsbcv3.zip saved ??? there are problems at this link…

  16. Andy says:

    Hi!

    Could anyone please put a new link instead of http://blog.mahalo.com/hackaday/howto/wsbcv3.zip

    This one doesn’t work as some other people already mentioned it…

    Thanks in advancE,
    Andy

  17. Ian says:

    The link to the file on my personal site should still work:
    whereisian (dot) com (slash) files (slash) wsbcv3.zip

  18. sqwale says:

    I was wondering if I could get the schematics of the PCB so I could create my own PCB from your design, I will give credit where its due and reference appropriately if you give me the details.

  19. Andy says:

    Hi Mate!

    You find everything in the zip file on the site which Ian mentioned in the message before yours…
    After solving the puzzle in Ian’s message the URL is..:
    http://www.whereisian.com/files/wsbcv3.zip

    Have fun Mate…!

    I am just about to put mine together…

    Credits to Ian

    Andy

  20. sqwale says:

    Is it just me or they have connected pin 5 (NC)of the ENC28j60 chip to the PIC after it explicity states do not connect? Further more the the pin 4 of the ENC28j69 is connected as well but PIC dose not appear to use it can someone explain to me why this is so? Please reply with my alias “sqwale” in the message.

  21. ian says:

    @sqwale –

    Pin 5 is NC in the current ENC28J60 revision, but when I wrote this article it was WOL or some other interrupt.

    Pin 4 is connected to allow further development and experimentation.

  22. sqwale says:

    Thank you very much that was helpful as I was confused.
    Regards Sqwale

  23. Sqwale says:

    Hi Ian I have gone through the errata of the pic24fj64ga002 and I still cant identify why My spi for my EEPROM dose not work Im aware you did not use and External EEPROM but may have come across something.

  24. ian says:

    Sorry, no, I never had a problem with external EEPROMs and the Microchip TCPIP stack.

  25. majorminer says:

    can it run a minecraft server

  26. zedy says:

    Web server source files doesn’t work. :(

    After patching FATHTTP.c project doesn’t compile, I managed to fix it by adding the definition of SD card inputs and outputs to the main.c

    But I encountered another problem, something is wrong with FATHTTP.c file. It gets stuck in some cycle between ENC28J60.c and FSIO.c when requesting web page from server.

    I’ll be very grateful if someone can send me a functioning project source files to my email erzzzz@inbox.lv

  27. pascal says:

    Hello everyone,

    This is a nice project , but i have a question.
    What kind of programmer do i need to program the pic24….? pickit2, galvawisp mark2 or else??

  28. Anonymous says:

    all the rasberry pi fanboys never even saw this shit.

  29. andy says:

    Hello Guys!

    I am now trying to compile the web server code…

    But I don’t even know what version of Microchip Application Libraries were used with the project at the time when it was in progress…

    Could anyone send me any info about it please!!

    I downloaded the 2009-11-18 library package and I am trying to apply the HTTP patch to the HTTP.c file but it feels very much like if it was the wrong version library file…
    The things I need to delete and insert feel sometimes like if they didn’t belong there…

    Thanks..
    Andy

  30. jimmy says:

    is it still possible to download the whole project somewhere? the link is not working anymore.
    congrats for the job ,
    thanks

  31. CarlosH says:

    sorry guys, I haven’t seen all posts… there’s a link still working, this one http://www.whereisian.com/files/wsbcv3.zip

  32. Mmadmac says:

    What version of stack was this project built with? I’m experiencing some major trouble porting it.

  33. Jakob says:

    The project archive download-link is down!
    Can someone please reupload the files?

    Thanks,
    Jakob

  34. Áron Fehér says:

    Nice project! I see that since 2008 there hasn’t been any comment activity, still, I have a question. The 5th pin of the enc28j60 is specified as NC (reserved pin- leave disconnected) in the datasheet, but in the schematic is used as WOL. Which is the correct configuration, cause I can’t decide

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 94,571 other followers