How-To: Web Server On A Business Card (Part 2)

This mini web server is slightly smaller than a business card. There are a lot of tiny one-board servers out there, but this is probably the smallest you can etch and solder at home. Unlike many embedded web servers, files are stored on a PC-readable SD card, not in a difficult-to-write EEPROM. Read on for the web server design, or catch up on PIC 24F basics in the previous article: Web server on a business card (part 1).

Concept overview

The goal of this project is to build a web server on a business card that serves web pages and files from a FAT formatted SD card. The server is based on a PIC 24F that connects to a TCP/IP network using the ENC28J60 ethernet MAC/PHY. Network layers and low-level services, such as DNS and DHCP, are handled by the Microchip TCP/IP stack. A FAT 12/16/32 formatted SD card contains web pages and files. A very simple HTTP server ties everything together by handling page requests on port 80, searching the SD card for requested, and serving them with the correct content type.

Hardware

Microcontroller (Microchip PIC 24FJ64GA002)

The brain of the server is a 16-bit PIC 24FJ64GA002 (IC1), a 28pin microcontroller available in several hobbyist friendly packages. Check out our PIC 24F introduction for more about working with this chip.

PIC 24Fs operate between 2 and 3.8volts, which is perfect because the ethernet chip (IC2) and SD card both run at 3.3volts. This chip has 8K of RAM, plenty for the TCP/IP stack and a few K for working with a full FAT file system. The 24FJ64 has two SPI modules, so the SD card and ethernet IC each get a dedicated data bus.

The PIC processor core operates at 2.5volts, and requires a 10uF capacitor (C2) for the on-chip voltage regulator. The datasheet specifies a tantalum capacitor, but we used a low-ESR electrolytic in a prototype without incident. Every power pin needs a 0.1uF decoupling capacitor (C4,5).

The internal 8MHz oscillator provides a 32MHz clock source with the 4x PLL multiplier enabled. We’re also using an external 32.768KHz crystal (Q1) with 2 x 27pF capacitors (C17,18) to enable the real time clock calendar.

Programming connections are brought to a header (SV1). We chose to use programming pin pair three (PGx3). The master clear and reset (MCLR) function is enabled with a 2K resistor (R1) from V+ to the MCLR pin. Optionally, add a button (S1) from MCLR to ground for a manual reset switch.

Ethernet connection (ENC28J60)

An ENC28J60 (IC2) handles the network physical connection (PHY) and MAC layer. The ENC28J60 needs a number of support parts beyond the typical 0.1uF decoupling capacitors (C6,7,9,10). A 25MHz crystal (Q2) and 2 x 27pf capacitors (C15,16) provide a clock signal. The internal core voltage regulator requires a 10uF tantalum capacitor (C1), but an electrolytic capacitor also worked fine. Two LEDs (LED1,2) with 330ohm resistors (R2,3) display link and data status.

A bias resistor (R12) is required; the value will depend on the ENC28J60 version you’re using. Current chips should be B5 (PDF) or B7 (PDF), and require a 2.32K 1% resistor.

The PHY I/O portion specifies 4 x 49.9ohm 1% resistors (R8-11), and a ferrite bead (L1).

The most difficult-to-find part for the ENC28J60 is the correct RJ-45 jack with integrated magnetics (RJ1). We used a J1006F21 PulseJack from Pulse Engineering. Be sure to check the pin configuration and connections if you use a different jack, they will probably be different than ours. A Cadsoft Eagle part library for the JP1006F21 is included in the project archive. This was a $4 part, but it’s gone up to $7. If you know of other jacks that work we’ll add them here.

microSD card

We used a microSD/transflash card in this design because SD cards waste a lot of board space under the holder. microSD cards are smaller versions of SD cards with the same data interface, and most come with an adapter for use in standard SD card readers. The card needs a holder (SD1) and a 0.1uF decoupling capacitor (C8).

If you want to use a full-size SD card, take a look at our version one prototype in the project archive. We used Alps SD card holder #SCDA1A0901. Unfortunately, this part is has been discontinued and we’ve yet to find a suitable replacement. Don’t try #SCDA5A0201, that’s for sure. If you have a favorite, we’ll add it here. Sparkfun has one, and a matching Cadsoft Eagle part library.

Power supply

An adjustable LM317 voltage regulator (IC3) is set to 3.3volts using a 390ohm (R6) and 240ohm (R7) resistor. We considered several 3.3volt regulators, but nothing was cheaper than a LM317 and two resistors. There’s a 0.1uF decoupling capacitor (C13,14) and a 10uF capacitor (C3,19) on both sides to help support the power hungry Ethernet transceiver. The LM317 will output 3.3volts from an input of 5 to 20volts+, but it gets really hot with greater than 9volts supply. The specified input capacitor is only rated 16volts, so consider an upgrade if you plan to use a supply greater than about 9volts.

For the first time ever, we incorporated a power jack (J1) into a design. A jack with a 2.1mm diameter internal pin seems to be the most common DC connector. We used a cheap through-hole DC power jack, like SparkFun #PRT-00119 or Mouser #163-7620-E. It mates with a plug like Mouser #1710-0721.

Circuit board

The PCB (full size placement .png) was designed in Cadsoft Eagle 5.0. Freeware versions are available for all major platforms. Renderings were done with Eagle3D, beta version. Schematic and board files are included in the project archive (ZIP).

We designed the project with large SOIC chips and 0805 surface mount (SMD) parts, but haters can rest assured that chips are available in a through-hole package. We prefer to use SMD parts because the resulting circuit boards are smaller, cheaper, and faster to produce. 0805 parts are dirt cheap, and easy to solder with a normal iron. Don’t expect this project to work on a breadboard, there’s probably too much capacitance for this circuit.

We took full advantage of the PIC’s programmable pin placement to get the simplest trace routings possible. Just four jumper wires are needed on an otherwise single-sided board.

The traces are large and clean, DIY toner transfer boards should be easy. We made our PCB using an inkjet printer transparency mask over an UV sensitive circuit board.

In addition to the final design, the project archive contains our v1 prototype design. The prototype uses a full size SD card (SCDA1A0901) and all electrolytic 10uF capacitors. We also put the RJ45 Ethernet jack on a daughterboard to better accommodate different pinouts.

Partslist

# Count Part Size Mouser#
IC1 1 PIC 24FJ64GA002-SO SOIC-28 579-PIC24FJ64GA002SO
IC2 1 ENC28J60 SOIC-28 579-ENC28J60-I/SO
IC3 1 LM317 voltage regulator D2Pack 511-LM317D2T-TR
C1-3 3 10uF tantalum capacitor A case 74-293D106X96R3A2TE3
C4-14 11 0.1uF capacitor 0805 80-C0805C104M5R
C15-18 4 27pF capacitor 0805 140-CC501N270J-RC
C19 1 10uF capacitor 0805 647-UWF1C100MCL1GB
R1 1 2K0ohm resistor 0805 292-2.0K-RC
R2-6 5 390ohm resistor 0805 71-CRCW0805-390
R7 1 240ohm resistor 0805 71-CRCW0805-240
R8-11 4 49.9ohm 1% resistor 0805 71-CRCW0805-49.9-E3
R12 1 2K32ohm 1% resistor 0805 71-CRCW0805-2.32K-E3
L1 1 Ferrite bead 0805 81-BLM21BB600SN1D
LED1-4 4 LED 0805 645-598-8110-107F
Q1 1 32.768KHz crystal SMD 695-CM200S-327KF-U
Q2 1 25MHz crystal HC49 815-ABLS-25-B2
RJ1 1 Pulse J1006F21 673-J1006F21
S1 1 Tactile switch DTS-6 101-0164-EV
SV1,2 11 .1″ male pin header 571-41033290
J1 1 Power jack 163-7620-E
SD1 1 microSD card holder SparkFun: PRT-00127

Firmware

Three firmware examples are included in the project archive [zip]. The examples compile with Microchip’s demonstration C30 compiler. Learn more about working with the PIC 24F in our previous article: Web server on a business card (part 1). MPLAB isn’t great about project portability, you may need to locate all the project files again if your path doesn’t match the ‘c:wsbc’ format that we used.

FAT12/16/32 disk library

Our first step was to get the FAT library reading from a SD card. FAT 12/16/32 are simple disk storage formats that work with PCs, MACs, digital cameras, music players, and other electronics. Here’s our favorite FAT tutorial/teardown (PDF).

Microchip’s FAT 12/16/32 library gives us simple functions for working with SD cards. The included demo application creates some files and directories to demonstrate each function. Here’s how we configured it to work on our custom hardware, you can find these changes by searching for the tag ‘HACKADAY’ in the code:

  • HardwareProfile.h assigns actual PIC hardware to generic references in the code library. For the SD card this is an SPI interface, and pins for chip select and card detect. First, we deleted all the unused hardware profiles to make the code more manageable. Next, we configured the FAT library to communicate with the SD card using an SPI module (line 132). Finally, we defined the SPI pin assignments (line 152). Pin setup is shown in the table below.
    Pin Port
    Chip select B0
    SD card detect A2
    SPI clock B2
    SPI MOSI B1
    SPI MISO B3
  • Demonstration.c. On line 48 we set a custom oscillator fuse configuration, as described in our PIC 24F introduction. This is also the logical place to configure pin assignments with peripheral pin select (line 63).
  • FSConfig.h. This file enables various components of file system library, affecting the amount of memory and program space used. A read-only library is very small, a full write configuration is bigger. We didn’t have to make any changes for the demonstration, but this is an important file to note.

At first, the library failed to recognize our SD card. It only supports disks with a master boot record (MBR). Windows XP formats SD cards as a DOS disk: a single partition with no MBR. To verify this, open a Windows-formatted disk with a utility like HxD and inspect sector 0 of the physical disk. Byte 446 should be the location of the first MBR partition entry, but instead it’s the NTLDR executable code.

To format the disk in the ‘correct’ FAT format, use a digital camera’s format function or a utility like Panasonic’s SD card formatter. We also considered using a different FAT library that reads DOS disks, like DOSFS, or adding similar features to the Microchip firmware.

TCP/IP stack

Microchip’s free TCP/IP stack performs the convoluted configuration and networking functions needed to run a web server. You can read all about the stack in various application notes and documentation. Wikipedia is our favorite TCP/IP learning resource; we wrote our first TCP/IP stack using only Wikipedia.

Microchip’s TCP/IP stack used to be messy and confusing. Now it’s just confusing. The last few versions of have improved considerably in code clarity and structure. Here’s what we did to to configure the base TCP/IP stack example for our hardware, you can find these changes by searching for the tag ‘HACKADAY’ in the code:

  • HardwareProfile.h assigns actual PIC hardware resources to generic references in the code library. We added our custom oscillator configuration (line 68), and configured the server status LED to use the LED attached to PORTB7 (line 83).  We defined the SPI interface to the ENC28J60 as follows (line 116):
    Pin Port
    Reset B8
    Chip select B9
    SPI clock B10
    SPI MOSI B11
    SPI MISO B12
    Wake on lan B13
    Interrupt B14
  • MainDemo.c. We eliminated a bunch of unused code, and added the peripheral pin select configuration code to the InitializeBoard() function (line 332).
  • TCPIPConfig.h defines the TCPIP stack components included in a compile. We’ve enabled DNS, DHCP, the IP announcer, and the ping server (line 56):
#define STACK_USE_DNS            // Domain Name Service Client
#define STACK_USE_DHCP_CLIENT    // Get DNS automagically
#define STACK_USE_ANNOUNCE       // Microchip Ethernet Device Discoverer
#define STACK_USE_ICMP_SERVER    // Enable the PING server

After loading this firmware, we’re ready to connect the server to a network for the first time. During initialization, the TCP/IP stack negotiates with the network router for an IP address using DHCP. We need to know this address to communicate with the device. If the device had a screen we could display the IP address, but instead we use the MCHPDetect.exe utility from Microchip.

When the TCP/IP stack finishes initializing, it broadcasts an announcement packet to port 30303 of all locally connected computers. MCHPDetect extracts the IP address from these packets. A new announce packet is sent on every PIC reset.

It’s also possible to read the IP address directly from memory with a debugger. The address is stored in the AppConfig.MyIPAddr variable, the .byte form follows the standard x.x.x.x IP notation.

Once we have the IP address, we can ping the server and test its responsiveness.

If ping shows high latency or malformed packets, you can use Wireshark to inspect network traffic at the byte level. Unless you’re in Germany, because it might be criminal.

Building the custom HTTP server

The custom web server looks for requested files on the SD card, and sends them with the correct content type. We used the Microchip HTTP example server v1 (HTTP.c) as a base for our FAT file server (FATHTTP.c).

Microchip’s HTTP server used a simple file system called MPFS to index web pages on an EEPROM chip. We replaced calls to MPFS functions with calls to functions in the FAT library (see the HTTPProcess and Sendfile functions in FATHTTP.c). Our changes demonstrate the concept as simply as possible, without adding confusing pointers and other handy C obfuscations. The code leaves a ton of room for improvements, have at it. File writes are disabled in the default compilation, but there’s enough program space to enable them if you want to write to the SD card (see FSConfig.h).

It’s necessary to registered our custom FATHTTP server with the rest of the TCP/IP stack. We did a search and replace for the original HTTP server components, and added calls to our new FATHTTP server as needed. That turned out to be these places:

  • TCPIPConfig.h. First we inserted some definitions that enable the FATHTTP server (line 70), and added a TCP socket for the FATHTTP server (line 248).
  • TCPIP.h. Next, we added FATHTTP to the list of services that require the TCP/IP stack (line 170) and then included the necessary headers (line 351).
  • StackTSK.c. We added the FATHTTP server initialization (line 138) and processing (line 340) functions to the list of TCP/IP stack tasks.
  • Helpers.c. We also needed to include a few helper functions for working with URLs (line 259).

At long last, it’s time to put some files on an SD card and test this thing. Make sure your files follow the 8.3 file name format. The project archive contains a sample website with a test image and zip file.

After grabbing the server’s IP address with MCHPDetect, we pointed a browser at it. The IP address entered alone will redirect the browser to index.htm, whether or not it exists. Web pages and images stored on the SD card display in the browser, but unknown binary types trigger a download prompt.

Taking it further

We see a lot of potential projects using this tiny web platform.

  • Add hooks in the FATHTTP.c source for special URLs that trigger events or configure pins.
  • Build a remotely accessible data logger. Use the extra pins to read sensors and log data to the SD card. Logs are retrievable from a web browser, or directly from the FAT readable SD card.
  • Get remote access to an ancient serial terminal or BBS, optionally log the console output. Use two external pins as a serial port, and forward commands from the Internet using Microchip’s Telnet server and Ethernet-to-serial bridge examples.
  • Your suggestions?

Next time, we’ll use the mini server to make an Internet connected, electronic indoor graffiti wall. This will be an interactive project where everyone can contribute graffiti and animations on-line.

Schematic, board, and firmware files are included in the project archive (ZIP). Use the freeware version of Cadsoft Eagle to view the schematic and PCB. The firmware is written in C, and compiled with the Microchip demonstration C30 compiler.

140 thoughts on “How-To: Web Server On A Business Card (Part 2)

  1. this is genius m8 I was wondering is it possible install on this cute thing mysql and php is that possible could it work if more then 100 ppl connect to a server could that be stable or it will need more ram and kind of cpu ?!?!?! could it be used as some server that can support blog, forums or cms systems like joomla?

  2. For anyone wondering, these parts are all easily available for probably 20 bucks or so. I’m sure if you tried hard enough you could get samples for free. As for a PCB that would be about 60 bucks. All the hard work is done for the project, a very in depth post nice work ian.

  3. I’m not sure where Kurf is getting numbers from. I got my PCB in a batch order for $5/ea and the parts (minus uSD) cost about $40. Though I payed more like $70 cause of a mistake I made regarding shipping methods.. The price I got the PCBs for was great… I think you could make one yourself for ~$10 if you already have some common tools.

  4. I need this with added function of analog sensor reading input over 1 wire network. Anybody interested?

    Read Maxim sensor (DS18B20) x 2 x time interval
    Pulse read x 1
    preformatted Web page – plug in data
    more info if interested.

  5. Anyone still got some in or willing to send to UK ?

    Would be even more interested if it could easily be modified to take a full size SD card as the local retailer is letting them go for a joke.

  6. Got mine working first go, nice project.
    I would love to see a version of the bus pirate firmware to run on this board.

    Also i was looking at ways of gaining extra I/O ,
    It seems that pin 5 on the enc28j60 isnt a required pin now according to the data sheet. Can you think of any reason to leave it connected?

  7. This might be a stupid question but I’m going to ask it anyways. I downloaded the wsbcv3.zip file and opening the 24FJ-BCserver-V3.brd file in Eagle 5.3 does not show the large ground trace. Anyone know why that is? What am I missing?

  8. @peter

    The Bus Pirate relies on the PPS crossbar, but I think only a few of the external pins on this project have that feature. All your external pins would need to come from PORTB.

  9. Hi,

    I am currently working on trying to get the HTTPGetVar and the HTTPExecCmd commands working again in the fathttp.c , those functions seem to have been removed. Does anyone have a fixed version before i start hacking my way though this? As it would save me alot of time.

    Oh and the Bus pirate idea. ah well. worth a try.

    Peter

  10. I was in on a purchase “over-seas” and just received back a partially completed board with a spare PCB, and some other parts. I am looking for someone to finish building it for me and am willing to pay a reasonable amount to take what is currently useless to me and make it useable. I will also pay for any parts that might be missing. ps. whoever does this can have the other PCB as part of the deal.
    email to stuffit.now gmail com

  11. I don’t see any places in the code that enable pull-up resistor for SDCD pin (and WE pin if full size card used)

    Also MDD_SDSPI_MediaDetect routine called before MDD_SDSPI_InitIO init routine that sets SDCD pin as input and (missing) enables pullup.

    Or am I mossing something?

  12. I think it’s awsome idea having a handheld server that fit in a pocket. I think it lacks mobility of serving other devices the bluetooth and wireless connection would be better. Would it be poosible to have multiple micro sd card holders for more space to store stuff and be great for just holding spare sd cards. I’m not knockin on da hack but it can be better.

  13. @ Peter
    did you get the HTTPGetVar and the HTTPExecCmd commands working again in the fathttp.c ? Pleas contact me on perostoka@gmail[dot]com, i would like to ask you a advice!

    Paolo

  14. I did get the HTTPExecCmd command working. Now i can control the LED0 and LED1 on the board via the web page, but i’m still trying to get the httpgetvar working. Send me a email (perostoka@gmail[dot]com), ‘ll talk and we can share our knowledge.
    Paolo

  15. Were can i find the programming interface (api)
    to access get and post variables provided by user through form postbacks and url query strings?

    Would there be a sample online that has a simple example.. i’ll be able to extend it after that.

    -lm

  16. Board built and ready to program PIC, need help with ICSP (which circuit to use..I found the various messages very confusing )
    Any helpful pointers appreciated…..can’t afford the Microchip (commercial) programmers!
    Thanks Dave

  17. does anyone have an extra bare pcb? i noticed that some of you received and extra pcb with out components and i am in need of one. contact me at lunchboxrts at gmail dot com

  18. Well the beast is up and running, due to our local ISP we had to rework the code to run on port 81 and utilize a redirector page to access it cos of dynamic IP suppied by said provider (Telus) here in Vancouver area of Canada.
    Check it out live at: http://microserver.yolasite.com/
    Thanks to Ian Lesnet and Hack A Day for hosting his wonderful little project.
    Dave (VA7DB)

  19. Dear Ian,

    I’d like to know what is the performance of the web server, how much time does it take to download for example a 1Mb zip, or jpeg, or something. If someone else knows the answer please let me know!

    Thank You!

    @dave If I try to open the web page on your site it says “Not found.”
    Although I can ping that address from command line.

  20. Hi Boris,yes it appears to lock up occasionally, not sure of the reason yet, will investigate with Wireshark…..as for the load speed in your question above, the unit is fairly slow due to it’s limited capacity. I kept my pics to a few dozen Kb as larger Mb files take a minute or so.
    great little project though, enjoying it thoroughly.
    I’ll reset it now, try again….may have to add a regularly timed reset into the code if we cannot find the cause of the lock ups.
    Dave

  21. Just a note for anyone attempting this project as-is: SparkFun changed the microSDcard holder they sell, but didn’t change the part number. This has caused problems for a few people. SparkFun has the footprint for their new SD card holder so you can update your PCB, but it’s a more difficult part to hand-solder.

Leave a Reply to zilymCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.