I know this is a very long email so bear with me. None of this is very
hard for most programmers.
As a lot of you may know, I've been working on adding support for UPnP
into Gaim. As of yet, I have not written any code into Gaim but I have
made many working example apps and they all work. I'm trying to make
gaim automaticly setup port forwarding with any UPnP capable router, so
when you and your buddy are both behind a different NATs you can still
do your file transfers, direct connects, and whatever else you can think
of. I just don't believe that I'm going to have much time time
personally to do any coding since my vaction time is over and I've got
to much work to do. So, I have documented everything I have learned and
written a very basic, step by step cheat sheet for any developer to pick
up where I left off. I would love to see this get written as a patch for
Gaim but it would work great as a plug-in also.
First off, lots of routers have support for UPnP IGD now like most
consumer size routers and firewalls (like most Linksys, DLink, Netgear,
2Wire, etc...), Internet Connection Sharing in Windows XP, and any
Linfux router running UPnP IGD for Linux (http://upnp-igd.sf.net). The
Windows clients for AIM, MSN, and Yahoo already take advantage of this.
I'm not trying to add full UPnP support so its not as complicated as
some would think. In Windows platforms, you can make a call to a COM
object or an API and Windows will do all the work for you, but for Linux
its a bit more complicated becuase you have to go down to the protocol
itself to make the calls. There has been work to impliement the server
side of the Internet Gateway Device (IGD) spec under Linux
(http://linux-igd.sf.net), but the client side libraries to take
advantage of UPnP in linux are lame and way over-bloated to be useful
(http://upnp.sf.net). Everyone doing it in Linux apears to have used the
tool created by Intel that generates a bunch of C so you can just drop
it in and go. Unfortunatly their code creates its own web server, its
own XML/SOAP handling framework, and a gaint over bloated set of mostly
unused APIs that don't do much more then be huge (most of this is for
the event framework of SOAP but we don't need to impliment any type of
callbacks).
I personally believe the UPnP protocol is just another fad and Microsoft
promotional protocol. It's way over bloated and does a lot of other
stupid functions that serve no pratical purpose, like trying to connect
your TV media players to your clocks to your toothbrushes and to
whatever else has the ablity to be in wired in together with it.
However it does have one great part. They have the first standarization
of any kind between these types of devices to be able to preform some of
the basic functions provided by most of these devices. Like for example,
you can call simple things like setting up port forwarding for NAT
transversal, querying your external ip of your router, and getting a
call back event when your connection between your provider and your
router goes down, all from inside your code.
I've done all the research on all the protocol documentation, analyzed
and documented packets used by other UPnP programs like MSN and AIM, and
I was able to reproduce working examples that work with 3 different UPnP
routers (2 different Linksys models and 1 Netgear) and get it to work on
both linux and windows.
I believe that it doesn't need more then a few hundred lines of code at
the very most and about 90% of that should be just string playing and
defining a bunch of very long but static multiline strings for the funny
HTTP headers and all the XML/SOAP bodies for the messages.
On start gaim will need to:
* broadcast an http-like message over UDP to discover your UPnP devices.
* listen for a possible http-like response from your router over that
same UDP port if any UPnP devices exist.
* make a simple http request to URL in the location tag in that last UDP
message, and pull a URL for the specific function you want out of the
returned XML.
When your ready to open the port just do a simple little http post to
that URL form before for each port you want to open. Your router will
setup up port fowarding for you automaticly.
The entire procedure requires no configuration or interaction by the
user to use UPnP, so maybe we don't have to hear as many complaints by
lusers about their file transfers not working behind their firewalls.
The only added strain on anything is that when Gaim is ran inside a
private IP range or in the default configuration address range, it will
send a single UDP broadcast out (maybe two since its UDP) when it
starts. It will then have to listen (only maybe for 1 minute or so) for
a possible response that could come back from your UPnP devices.
The only security risks I can think of is that you end up listening for
a UDP packet comming from an unknown source and even though its hard to
see any issues with this, if left unchecked someone could spoof they are
a UPnP device and have you download from them.
Someone asked me what UPnP IGD port fowarding does when mulitple
applications request to use the same port. Just to anwser people's
questions on that one thing, UPnP IGD devices usally just overwrite an
existing port mapping with the new one. It also allows you to do IP
filtered port forwarding, meaning you can optionaly pass the IP address
of the device that you know will be contacting you. So if you want to
transfer a file with your friend, and your IM client knows your friends
IP, but your mom is in the other room is doing a bunch of file transfers
with some of her friends on the same port with MSN messanger, you can
both still connect to each other. I don't know about the specifics on
getting this to work with the remote IP specific version of this with
each protocol because you have to be able to get the IP public address
of the other person before he connects to you and some protocols may not
have this ablity, but if they do you have already got AIM and MSN beat
on that feature without trying.
I should mention also that UPnP IGD allows you to set a timeout on the
port mapping but I haven't seen applications even use it. They have all
just been passing 0, so their port mappings never expire (even though
the WRT54G from Linksys will automaticly expire mappings set for 0 in 24
hours because it doesn't have support for a maximum number of mappings
like most devices do).
The details on how to impliment UPnP can be found below:
--------------------
First off, you should always send 2 UDP broadcasts to
239.255.255.250:1900 in this http like format.
(in C String Syntax just because I'm a nice guy)
"M-SEARCH * HTTP/1.1\r\n"
"Host:239.255.255.250:1900\r\n"
"ST:upnp:rootdevice\r\n"
"Man:\"ssdp:discover\"\r\n"
"MX:3\r\n"
"\r\n"
"\r\n"
XP sends this when it joins a network:
M-SEARCH * HTTP/1.1
Host:239.255.255.250:1900
ST:upnp:rootdevice
Man:"ssdp:discover"
MX:3
Its a good idea to send two a few seconds apart becuase UDP isn't a nice
protocol for making sure everything has found its final destination...
You should recieve a bunch of udp responses from all the UPnP devices on
your network so you should maybe have a thread and listening socket for
any response you might get later. The responses will come back to on the
same UDP port used to sendout the first UDP query.
Its a good idea maybe to keep that thread open because it may be useful
later when you want to call your port forwarding requests unless you
plan on sending it only right before known communications on that port
are just about to start.
The response message you want most from all the devices, should look
something like this (from my Linksys BEFSR41 firmware v1.05.00):
NOTIFY * HTTP/1.1
HOST:239.255.255.250:1900
Cache-Control:max-age=120
Location:http://192.168.1.1:5678/rootDesc.xml
NT:urn:schemas-upnp-org:service:WANIPConnection:1
USN:uuid:upnp-WANConnectionDevice-1_0-0090a2777777::urn:schemas-upnp-org:service:WANIPConnection:1
NTS:ssdp:alive
Server:NT/5.0 UPnP/1.0
or like this (from my Linksys WRT54G with my own custom firmware based
on version v2.0.2)
HTTP/1.1 200 OK
ST:urn:schemas-upnp-org:service:WANIPConnection:1
USN:uuid:000625d8-095f-0006-25d8-095f0232aa18::urn:schemas-upnp-org:service:WANIPConnection:1
Location:
http://192.168.1.1:5431/dyndev/uuid:000625d8-095f-0006-25d8-095f0032aa18
Server: Custom/1.0 UPnP/1.0 Proc/Ver
EXT:
Cache-Control:max-age=1800
DATE: Mon, 29 Nov 2004 16:58:01 GMT
The part of the message that is the most valuable is the last part of
the USN value just after the "urn" tag. You can use that to filter out
the specific device we want to talk to. We want a device that follows
the Internet Gateway Device (IGD) specification in the UPnP spec, so you
want to look for a value that says
"schemas-upnp-org:service:WANIPConnection:1".
You will then need to make a basic HTTP request back to the value that
was passed to you as the "Location" tag to get some more information. It
should be as simple as this:
GET <path to description> HTTP/1.1
HOST: <host for description>:<port for description>
ACCEPT-LANGUAGE: en
You can also pass a referer or an acceptable language tag but I don't
believe its even nessisary. You should recieve an XML discription with
what services are provided by your device it should look something like
this:
<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<URLBase>http://192.168.1.1:5678</URLBase>
<device>
<deviceType>urn:schemas-upnp-org:device:InternetGatewayDevice:1</deviceType>
<presentationURL>/index.htm</presentationURL>
<friendlyName>Linksys BEFSR41/BEFSR11/BEFSRU31</friendlyName>
<manufacturer>Linksys Inc.</manufacturer>
<manufacturerURL>http://www.linksys.com</manufacturerURL>
<modelDescription>Internet Access Server</modelDescription>
<modelName>Linksys BEFSR41/BEFSR11/BEFSRU31</modelName>
<UDN>uuid:upnp-InternetGatewayDevice-1_0-0090a2777777</UDN>
<UPC>00000-00001</UPC>
...
On some devices, they might provide a lot of functions in this xml file
so sometimes it can be quite long sometimes spaning multiple packets so
a basic http function is good. You are looking for a set of tags looking
something like this:
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANIPConnection:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANIPConnection</serviceId>
<controlURL>http://192.168.1.1:2468//WANIPConnection</controlURL>
<eventSubURL>/WANIPConnection</eventSubURL>
<SCPDURL>/WANIPCn.xml</SCPDURL>
</service>
</serviceList>
or like this on my WRT54G:
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANIPConnection:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANIPConn1</serviceId>
<controlURL>/uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1</controlURL>
<eventSubURL>/uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1</eventSubURL>
<SCPDURL>/dynsvc/WANIPConnection:1.xml</SCPDURL>
</service>
</serviceList>
More specificly, you are looking for a "ServiceType" tag that equals
"urn:schemas-upnp-org:service:WANIPConnection:1".
After you have that, you want to remember the controlURL tag value for
later until you are ready to fire your port forwarding actions.
The specificiation for UPnP says "To provide firewalls and proxies
greater administrative flexibility, SOAP specifies that requests must
first be attempted without the MAN header or M- prefix. If the request
is rejected with a response of "405 Method Not Allowed", then a second
request must be sent using the MAN header and M-prefix. If that request
is rejected with a response of "501 Not Implemented" or "510 Not
Extended", the request fails. (Other HTTP responses should be processed
according to the HTTP specification.)"
For the attempt we will push the this http post request to the router:
POST <path of control URL> HTTP/1.1
HOST: <host of control URL>:<port of control URL>
CONTENT-LENGTH: <bytes in body>
CONTENT-TYPE: text/xml; charset="utf-8"
SOAPACTION:
"urn:schemas-upnp-org:service:WANIPConnection:1#AddPortMapping"
or if the that returns any error like stated in the spec then make the
request with a message like this:
M-POST <path of control URL> HTTP/1.1
HOST: <host of control URL>:<port of control URL>
CONTENT-LENGTH: <bytes in body>
CONTENT-TYPE: text/xml; charset="utf-8"
MAN: "http://schemas.xmlsoap.org/soap/envelope/"; ns=01
01-SOAPACTION:
"urn:schemas-upnp-org:service:WANIPConnection:1#AddPortMapping"
The body of both requests should be a basic SOAP message in the
following format:
<s:Envelope
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:AddPortMapping
xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
<NewRemoteHost>#string#</NewRemoteHost>
<NewExternalPort>#ui2#</NewExternalPort>
<NewProtocol>[TCP|UDP]</NewProtocol>
<NewInternalPort>#ui2#</NewInternalPort>
<NewEnabled>#boolean#</NewEnabled>
<NewPortMappingDescription>#string#</NewPortMappingDescription>
<NewLeaseDuration>#ui4#</NewLeaseDuration>
</u:AddPortMapping>
</s:Body>
</s:Envelope>
Replace these values with your own:
#ui2# = 2 byte unsigned integer
#ui4# = 4 byte unsigned integer
#boolean# = boolean
#string# = string
[TCP|UDP] = a "TCP" or "UDP" string
Real examples of it working are below:
You may get an error response back if something happens but I'm unsure
what they all look like. The documentation says it should always return
a XML SOAP type error but i've seen also sorts of information come back
durning an error. For some apps, the goal is "if it works, it works, and
if it doesn't, doesn't". I'm guessing that will be the general feeling
with the gaim implimentation since its not a requirement of gaim to
work, just a nice feature to add support for people with UPnP IGD
devices and are behind a NAT.
That I believe should be the minimum required to allow almost any
application to use the UPnP IGD port forwarding system. This specific
technique works on my system and I've seen it done by different software
like some closed source FTP programs, file share apps, and even Skype
for both Windows and Linux. AIM, Yahoo and MSN all call the Windows API
to handle it all, but with a little work we can do them one up.
See more on the UPnP spec at
http://www.upnp.org/download/UPnPDA10_20000613.htm
I really hope someone will finish what I have started. I will be here to
help if anyone needs anything or has any questions. I'm always check my
email about every hour and I can work on this late at night or on the
weekends. I have lots of more documents and packet dumps but I put all
the relevant information below.
I hope that helps anyone with any questions!
Zac Bowling
zac@...
Here is a sample of XP getting the discription XML from my WRT54G:
------ LOCAL ------
GET /dyndev/uuid:000625d8-095f-0006-25d8-095f0032aa18 HTTP/1.1
HOST: 192.168.1.1:5431
ACCEPT-LANGUAGE: en
------ WRT54G ------
HTTP/1.0 200 OK
SERVER: LINUX/2.4 UPnP/1.0 BRCM400/1.0
DATE: Mon, 29 Nov 2004 16:58:01 GMT
CONTENT-TYPE: application/octet-stream
Cache-Control: max-age=1
PRAGMA: no-cache
Connection: Close
<?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<device>
<deviceType>urn:schemas-upnp-org:device:InternetGatewayDevice:1</deviceType>
<friendlyName>Residential Gateway Device</friendlyName>
<manufacturer>Linksys Inc.</manufacturer>
<manufacturerURL>http://www.linksys.com/</manufacturerURL>
<modelDescription>Internet Access Server</modelDescription>
<modelName>WRT54G</modelName>
<modelNumber>v2.02.2</modelNumber>
<modelURL>http://www.linksys.com/</modelURL>
<UDN>uuid:000625d8-095f-0006-25d8-095f0032aa18</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:Layer3Forwarding:1</serviceType>
<serviceId>urn:upnp-org:serviceId:Layer3Forwarding:11</serviceId>
<controlURL>/uuid:000625d8-095f-0006-25d8-095f0032aa18/Layer3Forwarding:1</controlURL>
<eventSubURL>/uuid:000625d8-095f-0006-25d8-095f0032aa18/Layer3Forwarding:1</eventSubURL>
<SCPDURL>/dynsvc/Layer3Forwarding:1.xml</SCPDURL>
</service>
</serviceList>
<deviceList>
<device>
<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>
<friendlyName>urn:schemas-upnp-org:device:WANDevice:1</friendlyName>
<manufacturer>Linksys Inc.</manufacturer>
<manufacturerURL>http://www.linksys.com/</manufacturerURL>
<modelDescription>Internet Access Server</modelDescription>
<modelName>WRT54G</modelName>
<modelNumber>v2.02.2</modelNumber>
<modelURL>http://www.linksys.com/</modelURL>
<UDN>uuid:000625d8-095f-0006-25d8-095f0132aa18</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANCommonIFC1</serviceId>
<controlURL>/uuid:000625d8-095f-0006-25d8-095f0132aa18/WANCommonInterfaceConfig:1</controlURL>
<eventSubURL>/uuid:000625d8-095f-0006-25d8-095f0132aa18/WANCommonInterfaceConfig:1</eventSubURL>
<SCPDURL>/dynsvc/WANCommonInterfaceConfig:1.xml</SCPDURL>
</service>
</serviceList>
<deviceList>
<device>
<deviceType>urn:schemas-upnp-org:device:WANConnectionDevice:1</deviceType>
<friendlyName>urn:schemas-upnp-org:device:WANConnectionDevice:1</friendlyName>
<manufacturer>Linksys Inc.</manufacturer>
<manufacturerURL>http://www.linksys.com/</manufacturerURL>
<modelDescription>Internet Access Server</modelDescription>
<modelName>WRT54G</modelName>
<modelNumber>v2.02.2</modelNumber>
<modelURL>http://www.linksys.com/</modelURL>
<UDN>uuid:000625d8-095f-0006-25d8-095f0232aa18</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANIPConnection:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANIPConn1</serviceId>
<controlURL>/uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1</controlURL>
<eventSubURL>/uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1</eventSubURL>
<SCPDURL>/dynsvc/WANIPConnection:1.xml</SCPDURL>
</service>
<service>
<serviceType>urn:schemas-upnp-org:service:WANPPPConnection:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANPPPConn1</serviceId>
<controlURL>/uuid:000625d8-095f-0006-25d8-095f0232aa18/WANPPPConnection:1</controlURL>
<eventSubURL>/uuid:000625d8-095f-0006-25d8-095f0232aa18/WANPPPConnection:1</eventSubURL>
<SCPDURL>/dynsvc/WANPPPConnection:1.xml</SCPDURL>
</service>
</serviceList>
</device>
</deviceList>
</device>
</deviceList>
</device>
</root>
------ ------ ------
Here is a sample of XP setting up Port Forward:
------ LOCAL ------
POST /uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1 HTTP/1.1
HOST: 192.168.1.1:5431
CONTENT-LENGTH: 696
CONTENT-TYPE: text/xml; charset="utf-8"
SOAPACTION: "urn:schemas-upnp-org:service:WANIPConnection:1#AddPortMapping"
<s:Envelope
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:AddPortMapping
xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
<NewRemoteHost></NewRemoteHost>
<NewExternalPort>27360</NewExternalPort>
<NewProtocol>UDP</NewProtocol>
<NewInternalPort>8615</NewInternalPort>
<NewInternalClient>192.168.1.101</NewInternalClient>
<NewEnabled>1</NewEnabled>
<NewPortMappingDescription>msmsgs (192.168.1.101:8615) 27360
UDP</NewPortMappingDescription>
<NewLeaseDuration>0</NewLeaseDuration>
</u:AddPortMapping>
</s:Body>
</s:Envelope>
------ WRT54G ------
HTTP/1.1 200 OK
DATE: Mon, 29 Nov 2004 16:58:03 GMT
Connection: Keep-Alive
Server: LINUX/2.4 UPnP/1.0 BRCM400/1.0
Content-Length: 289
Content-Type: text/xml; charset="utf-8"
EXT:
<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><m:AddPortMappingResponse
xmlns:m="urn:schemas-upnp-org:service:WANIPConnection:1"></m:AddPortMappingResponse></s:Body></s:Envelope>
------ ------ ------
Here is another sample of XP deleting a port mapping (which souldn't be
required but its nice to document this):
------ LOCAL ------
POST /uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1 HTTP/1.1
HOST: 192.168.1.1:5431
CONTENT-LENGTH: 415
CONTENT-TYPE: text/xml; charset="utf-8"
SOAPACTION:
"urn:schemas-upnp-org:service:WANIPConnection:1#DeletePortMapping"
<s:Envelope
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:DeletePortMapping
xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
<NewRemoteHost></NewRemoteHost>
<NewExternalPort>41209</NewExternalPort>
<NewProtocol>TCP</NewProtocol>
</u:DeletePortMapping>
</s:Body>
</s:Envelope>
------ WRT54G ------
HTTP/1.1 200 OK
DATE: Mon, 29 Nov 2004 16:58:27 GMT
Connection: Keep-Alive
Server: LINUX/2.4 UPnP/1.0 BRCM400/1.0
Content-Length: 295
Content-Type: text/xml; charset="utf-8"
EXT:
<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><m:DeletePortMappingResponse
xmlns:m="urn:schemas-upnp-org:service:WANIPConnection:1"></m:DeletePortMappingResponse></s:Body></s:Envelope>
------ ------ ------
Here is another sample of XP unsucessfully getting the external IP
(since I'm using PPP/PPPoE you have to query that space to get the IP
adderess):
------ LOCAL ------
POST /uuid:000625d8-095f-0006-25d8-095f0232aa18/WANIPConnection:1 HTTP/1.1
HOST: 192.168.1.1:5431
CONTENT-LENGTH: 296
CONTENT-TYPE: text/xml; charset="utf-8"
SOAPACTION:
"urn:schemas-upnp-org:service:WANIPConnection:1#GetExternalIPAddress"
<s:Envelope
xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:GetExternalIPAddress
xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
</u:GetExternalIPAddress>
</s:Body>
</s:Envelope>
------ WRT54G ------
HTTP/1.1 200 OK
DATE: Mon, 29 Nov 2004 16:58:01 GMT
Connection: Keep-Alive
Server: LINUX/2.4 UPnP/1.0 BRCM400/1.0
Content-Length: 353
Content-Type: text/xml; charset="utf-8"
EXT:
<?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><m:GetExternalIPAddressResponse
xmlns:m="urn:schemas-upnp-org:service:WANIPConnection:1"><NewExternalIPAddress>0.0.0.0</NewExternalIPAddress></m:GetExternalIPAddressResponse></s:Body></s:Envelope>
------ ------ ------
|
