I have some questions however. I think that ndis intermediate drivers can only connect to ethernet adapters, no wan adapters (for the internet connections (ras,etc...) I didnt find in the msdn library info on how to connect a ndis driver to a remote access connection. Since Qos is very useful for the internet, you should look at it carefully.
What type of packet selection do you plan? is it application based (emule, http server , ftp server) ?
protocol based? (tcp/udp + port)
What kind of scheduling do you also plan? bandwidth limiting is a very poor algorithm. there are lots of QOS algorithms that are all best efforts.
some of them can even guarantee latency. lot of papers are discussing of these algorithms. you should take a look at http://citeseer.nj.nec.com/cs.
The simplest is of course the simple fixed priority algorithm which is very useful.
One typical need is also to make upload packets less important than download ack packets, so that download is not penalized by local servers...
I have no experience for windows ndis drivers, but i have read lots of papers about scheduling algorithms. Lots of them also have been implemented in the openbsd kernel. maybe it could cool to use them. ;)
I would be glad to help you.
Sophana
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I responded to your e-mail directly, but though for the sake of others, I would post my response here as well:
Sophana,
Thank you for such valuable information, and I would be happy to receive any help you have to offer.
First of all, I haven't really had a lot of time to work on Traffic Cop lately. I have three contract jobs and one other personal project that are, at this moment, consuming all my time.
I should be finishing one of the contract jobs on or about Christmas and the other personal project will hopefully be close enough to completion that I can set it aside in that time as well.
So, I'm hoping that sometime in early January, I can get back to Traffic Cop.
Right now, my thinking is something along these lines. All traffic would go through a round-robin style priority queue. The user would define filters. A filter might consist of a source address and/or port, destination address and/or port, a time of day, a day of week, and/or an application (such as eMule, Kazaa, etc), and so on. I haven't decided what all of the parameters of the filter will be yet.
For the time of day stuff, my thinking was, some people will want to prioritize certain traffic at certain times of day. For example, in an office, you might want to have certain restrictions that only apply during the day, but allow a free-for-all at night to maybe allow certain people to use all the bandwidth they can get to do large downloads.
With each filter would be a set of rules that are applied to that traffic. This is where things get a bit more complex. You can do hard limits on the bandwidth, or you can tell it to equally distribute the bandwidth in the filter to the clients that fall into it. You could do some sort of combination: For example: the maxmimum bandwidth for eMule is 64kbps, equally distributed among all clients.
You could also define limits as hard or soft. A hard limit would always be adhered to. A soft limit would only apply if there is other traffic consuming the rest of the bandwidth.
In addition, you'd be able to apply priorities to the filters. For example, you probably want to have a highest priority for something like PC Anywhere or VNC so that they're more responsive. Lower priorities would go to things like eMail, eMule or Kazaa, etc. Mid-priorities would go to things like HTTP and FTP traffic.
Using a round robin priority type queue where the longer something is queued, the higher its priority gets, allows you to ensure that all traffic eventually gets through, even if the lines are hogged up by high priority traffic.
I was thinking a 3-5 level queue would probably be sufficient.
Anyway, that's my current thinking and it's not set in stone yet. I have to completely rewrite my prototype driver from scratch and I need to do more research into it as well.
I'm not aware of the limitations on NDIS intermediate drivers that you mentioned. NDIS intermediate drivers are layered between the protocol driver and the NDIS miniport driver. So, it would slip in between TCP/IP and, say an ethernet card or your PPP driver, so it should work for any WAN driver. I think. I'm not positive on that.
According to the documentation, the following fall under the NDIS miniport category:
Ethernet
Token Ring
FDDI
LocalTalk
ARCNET
WAN (point-to-point, WAN cards, and connection-oriented WAN)
Wireless
ATM
IrDA
I haven't decided which protocols I will support. I'll definitely support TCP, UDP, and ICMP packets, but beyond that, I haven't really decided. I don't know if there's enough demand to worry about IPX/SPX or non-tcp based netbios traffic.
Thanks again for the information. I'll read through it as I'm sure it will provide me with more ideas and I appreciate any feedback you may have regarding my plans.
Pete Davis
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I was looking for such a driver!
I have some questions however. I think that ndis intermediate drivers can only connect to ethernet adapters, no wan adapters (for the internet connections (ras,etc...) I didnt find in the msdn library info on how to connect a ndis driver to a remote access connection. Since Qos is very useful for the internet, you should look at it carefully.
What type of packet selection do you plan? is it application based (emule, http server , ftp server) ?
protocol based? (tcp/udp + port)
What kind of scheduling do you also plan? bandwidth limiting is a very poor algorithm. there are lots of QOS algorithms that are all best efforts.
some of them can even guarantee latency. lot of papers are discussing of these algorithms. you should take a look at
http://citeseer.nj.nec.com/cs.
The simplest is of course the simple fixed priority algorithm which is very useful.
One typical need is also to make upload packets less important than download ack packets, so that download is not penalized by local servers...
I have no experience for windows ndis drivers, but i have read lots of papers about scheduling algorithms. Lots of them also have been implemented in the openbsd kernel. maybe it could cool to use them. ;)
I would be glad to help you.
Sophana
I responded to your e-mail directly, but though for the sake of others, I would post my response here as well:
Sophana,
Thank you for such valuable information, and I would be happy to receive any help you have to offer.
First of all, I haven't really had a lot of time to work on Traffic Cop lately. I have three contract jobs and one other personal project that are, at this moment, consuming all my time.
I should be finishing one of the contract jobs on or about Christmas and the other personal project will hopefully be close enough to completion that I can set it aside in that time as well.
So, I'm hoping that sometime in early January, I can get back to Traffic Cop.
Right now, my thinking is something along these lines. All traffic would go through a round-robin style priority queue. The user would define filters. A filter might consist of a source address and/or port, destination address and/or port, a time of day, a day of week, and/or an application (such as eMule, Kazaa, etc), and so on. I haven't decided what all of the parameters of the filter will be yet.
For the time of day stuff, my thinking was, some people will want to prioritize certain traffic at certain times of day. For example, in an office, you might want to have certain restrictions that only apply during the day, but allow a free-for-all at night to maybe allow certain people to use all the bandwidth they can get to do large downloads.
With each filter would be a set of rules that are applied to that traffic. This is where things get a bit more complex. You can do hard limits on the bandwidth, or you can tell it to equally distribute the bandwidth in the filter to the clients that fall into it. You could do some sort of combination: For example: the maxmimum bandwidth for eMule is 64kbps, equally distributed among all clients.
You could also define limits as hard or soft. A hard limit would always be adhered to. A soft limit would only apply if there is other traffic consuming the rest of the bandwidth.
In addition, you'd be able to apply priorities to the filters. For example, you probably want to have a highest priority for something like PC Anywhere or VNC so that they're more responsive. Lower priorities would go to things like eMail, eMule or Kazaa, etc. Mid-priorities would go to things like HTTP and FTP traffic.
Using a round robin priority type queue where the longer something is queued, the higher its priority gets, allows you to ensure that all traffic eventually gets through, even if the lines are hogged up by high priority traffic.
I was thinking a 3-5 level queue would probably be sufficient.
Anyway, that's my current thinking and it's not set in stone yet. I have to completely rewrite my prototype driver from scratch and I need to do more research into it as well.
I'm not aware of the limitations on NDIS intermediate drivers that you mentioned. NDIS intermediate drivers are layered between the protocol driver and the NDIS miniport driver. So, it would slip in between TCP/IP and, say an ethernet card or your PPP driver, so it should work for any WAN driver. I think. I'm not positive on that.
According to the documentation, the following fall under the NDIS miniport category:
Ethernet
Token Ring
FDDI
LocalTalk
ARCNET
WAN (point-to-point, WAN cards, and connection-oriented WAN)
Wireless
ATM
IrDA
I haven't decided which protocols I will support. I'll definitely support TCP, UDP, and ICMP packets, but beyond that, I haven't really decided. I don't know if there's enough demand to worry about IPX/SPX or non-tcp based netbios traffic.
Thanks again for the information. I'll read through it as I'm sure it will provide me with more ideas and I appreciate any feedback you may have regarding my plans.
Pete Davis