From: Koushik N. <kou...@gm...> - 2009-10-31 11:49:52
Attachments:
yaws-mime.patch
|
Hi, I had a requirement wherein changes to /etc/mime.types come into effect without requiring rebuilding yaws. So I patched yaws to use the ets table created while parsing mime.types for returning mime types and to rebuild the ets each time yaws starts. The patch is on Yaws-1.77 tree. Let me know if there are better ways of doing this. Also the idea of compiling in mime type mappings is for throughput ? Such behaviour causes deployment issues. Regards, Koushik Narayanan |
From: Steve V. <vi...@ie...> - 2009-10-31 13:59:03
|
On Sat, Oct 31, 2009 at 7:49 AM, Koushik Narayanan <kou...@gm...>wrote: > Hi, > > I had a requirement wherein changes to /etc/mime.types come into effect > without requiring rebuilding yaws. > Technically, you need to rebuild only the mime types module. So I patched yaws to use the ets table created while parsing mime.types for > returning mime types and to rebuild the ets > each time yaws starts. > > > The patch is on Yaws-1.77 tree. > > Let me know if there are better ways of doing this. > > Also the idea of compiling in mime type mappings is for throughput ? Such > behaviour causes deployment issues. > I guess that depends on how you use it. One, if you update the mime types and rebuild the mime types module, all you need to do is redeploy its beam file, not all of yaws. Second, if you start yaws in a manner that allows you to remsh or rpc into it, you can leave it running and just hot-reload the new mime types module on the fly without restarting yaws at all. This reload logic could easily be written in a small shell script, for example. Also, you might want to consider updating to 1.85, as I did a huge update on the MIME types recently, adding quite a few in the process. Perhaps with this newer version, the need to reload mime types will be greatly reduced or even eliminated for you. --steve |
From: Koushik N. <kou...@gm...> - 2009-10-31 15:12:24
|
Hi, On Sat, Oct 31, 2009 at 7:28 PM, Steve Vinoski <vi...@ie...> wrote: > > I had a requirement wherein changes to /etc/mime.types come into effect >> without requiring rebuilding yaws. >> > > Technically, you need to rebuild only the mime types module. > Yes. The scenario is I have a binary package of yaws given to a system administrator wants to add/remove MIME types. I guess that depends on how you use it. One, if you update the mime types > and rebuild the mime types module, all you need to do is redeploy its beam > file, not all of yaws. Second, if you start yaws in a manner that allows you > to remsh or rpc into it, you can leave it running and just hot-reload the > new mime types module on the fly without restarting yaws at all. This reload > logic could easily be written in a small shell script, for example. > Ok. But that would mean having an erl file generated at the deployment machine and compiled there. Though the erlang installations have both the compiler and the runtime installed, generally it wouldn't be required to have a compiler at a deployment machine. > > Also, you might want to consider updating to 1.85, as I did a huge update > on the MIME types recently, adding quite a few in the process. Perhaps with > this newer version, the need to reload mime types will be greatly reduced or > even eliminated for you. > I am stuck at 1.77 because, I am having problems in using the OTP http module to fetch data from yaws asynchronously in yaws versions greater than 1.77. And also I feel its not required to have a comprehensive list of all possible mime types at all deployments. Depending on the application specifying what is required would be better imho. Koushik Narayanan |
From: Steve V. <vi...@ie...> - 2009-10-31 16:30:44
|
On Sat, Oct 31, 2009 at 11:12 AM, Koushik Narayanan <kou...@gm...>wrote: > Hi, > > On Sat, Oct 31, 2009 at 7:28 PM, Steve Vinoski <vi...@ie...> wrote: > >> >> I had a requirement wherein changes to /etc/mime.types come into effect >>> without requiring rebuilding yaws. >>> >> >> Technically, you need to rebuild only the mime types module. >> > > Yes. The scenario is I have a binary package of yaws given to a system > administrator wants to add/remove MIME types. > > I guess that depends on how you use it. One, if you update the mime types >> and rebuild the mime types module, all you need to do is redeploy its beam >> file, not all of yaws. Second, if you start yaws in a manner that allows you >> to remsh or rpc into it, you can leave it running and just hot-reload the >> new mime types module on the fly without restarting yaws at all. This reload >> logic could easily be written in a small shell script, for example. >> > > > Ok. But that would mean having an erl file generated at the deployment > machine and compiled there. Though the erlang installations have both the > compiler and the runtime installed, generally it wouldn't be required to > have a compiler at a deployment machine. > Why does it have to be generated at the deployment host? A beam file is a beam file; it can be generated anywhere and run anywhere. You don't really need a separate compiler, BTW, given that the compile module is always available by default in Erlang/OTP unless you take it out. If you're changing mime.types, you can generate a new one from an erlang shell by calling mime_type_c:compile(), assuming the yaws ebin directory is in the erlang code path and your working directory is where mime.types resides. This will generate a new mime_types.erl source module. You can then compile that either using the c(mime_types) erl shell command, which will also load it for you, or by calling compile:file("mime_types") and then using the code module to load the object code. Either way, doing this from a shell script is quite easy. Also, this can all be done via a remsh to a running yaws instance, in which case generating and compiling the new mime_types module will load it right into your running web server. > Also, you might want to consider updating to 1.85, as I did a huge update >> on the MIME types recently, adding quite a few in the process. Perhaps with >> this newer version, the need to reload mime types will be greatly reduced or >> even eliminated for you. >> > > > I am stuck at 1.77 because, I am having problems in using the OTP http > module to fetch data from yaws asynchronously in yaws versions greater than > 1.77. And also I feel its not required to have a comprehensive list of all > possible mime types at all deployments. Depending on the application > specifying what is required would be better imho. > I guess I disagree. MIME types are registered with the IANA and are thus globally standardized and widely known. There's really no reason to subset them unless you're running in an embedded environment with memory limitations so severe that it takes too much space to hold MIME type definitions you'll never use. IMO the overall problem here is that by using just a subset of the MIME types, you then have to compensate for that at each deployment host. If you just deployed the right set to begin with, it wouldn't be an issue. When you say you have problems fetching data asynchronously via the OTP http module, I assume you're talking about the client side, so how can that be a yaws problem? Assuming the responses from yaws conform to RFC 2616, which I think is safe to assume otherwise we'd be getting all kinds of bug reports, the problem would seem to be in the client. Have you considered using the ibrowse client instead? There are a few known bugs in the httpc module. My overall advice is to 1) determine why your client has an issue, and fix that so you can move off 1.77 to a more recent yaws, and 2) to deploy a much more complete set of MIME types such as that found in 1.85, such that your sysadmins do not need to generate anything. If you need help fixing the client issue, please send the details of the problems you're seeing to this list, and we'll try to help as much as we can. --steve |
From: Koushik N. <kou...@gm...> - 2009-11-01 03:13:39
|
Hi, On Sat, Oct 31, 2009 at 10:00 PM, Steve Vinoski <vi...@ie...> wrote: > > > On Sat, Oct 31, 2009 at 11:12 AM, Koushik Narayanan < > kou...@gm...> wrote: > >> Ok. But that would mean having an erl file generated at the deployment >> machine and compiled there. Though the erlang installations have both the >> compiler and the runtime installed, generally it wouldn't be required to >> have a compiler at a deployment machine. >> >> > Why does it have to be generated at the deployment host? A beam file is a > beam file; it can be generated anywhere and run anywhere. > > You don't really need a separate compiler, BTW, given that the compile > module is always available by default in Erlang/OTP unless you take it out. > If you're changing mime.types, you can generate a new one from an erlang > shell by calling mime_type_c:compile(), assuming the yaws ebin directory is > in the erlang code path and your working directory is where mime.types > resides. This will generate a new mime_types.erl source module. You can then > compile that either using the c(mime_types) erl shell command, which will > also load it for you, or by calling compile:file("mime_types") and then > using the code module to load the object code. Either way, doing this from a > shell script is quite easy. Also, this can all be done via a remsh to a > running yaws instance, in which case generating and compiling the new > mime_types module will load it right into your running web server. > Agreed. But adding lines to a text file would certainly be easier. > > >> Also, you might want to consider updating to 1.85, as I did a huge update >>> on the MIME types recently, adding quite a few in the process. Perhaps with >>> this newer version, the need to reload mime types will be greatly reduced or >>> even eliminated for you. >>> >> >> >> I am stuck at 1.77 because, I am having problems in using the OTP http >> module to fetch data from yaws asynchronously in yaws versions greater than >> 1.77. And also I feel its not required to have a comprehensive list of all >> possible mime types at all deployments. Depending on the application >> specifying what is required would be better imho. >> > > I guess I disagree. MIME types are registered with the IANA and are thus > globally standardized and widely known. There's really no reason to subset > them unless you're running in an embedded environment with memory > limitations so severe that it takes too much space to hold MIME type > definitions you'll never use. > IANA allows usage of x- (extended) MIME types which are not required to be registered with it. Though its usage is discouraged in RFC2045, they are in widespread use. Also there are vendor specific mime types which are required when interfacing with specific clients. So it is very difficult to build up a set of all required MIME types. When you say you have problems fetching data asynchronously via the OTP http > module, I assume you're talking about the client side, so how can that be a > yaws problem? Assuming the responses from yaws conform to RFC 2616, which I > think is safe to assume otherwise we'd be getting all kinds of bug reports, > the problem would seem to be in the client. Have you considered using the > ibrowse client instead? There are a few known bugs in the httpc module. > Yes. Its a bug in the httpc module. Since development was done on 1.77 it was not noticed. Will be changing the http client soon after which will upgrade to the recent yaws release. > My overall advice is to 1) determine why your client has an issue, and fix > that so you can move off 1.77 to a more recent yaws, and 2) to deploy a much > more complete set of MIME types such as that found in 1.85, such that your > sysadmins do not need to generate anything. If you need help fixing the > client issue, please send the details of the problems you're seeing to this > list, and we'll try to help as much as we can. > Yes I will move to 1.85 soon. I will be shifting the http client to ibrowse or the http client from ETC. Koushik Narayanan |