Tinyproxy

File Release Notes and Changelog

Release Name: 1.5.0

Notes:
 INTRODUCTION
 ------------

tinyproxy 1.4.3 was released on November 21, 2001.  It was hoped that
the next release (in this case 1.5.0) would follow quickly, but
unforeseen events occurred.  In a way it was probably better that the
release cycle took so long since a tonne of bugs have been removed
from tinyproxy, and there has been a bunch of new functionality
added.  However, the plan is to release a new version of tinyproxy at
_least_ once a month.


 CHANGES
 -------

For a complete list of all the changes, please see the ChangeLog
included with this distribution.  Below is a brief summary of the most
important changes:

 * The DNS caching system has been removed from tinyproxy, since
   according the RFC 2616, an HTTP client should not cache an address
   if it does not respect the TTL value.  Additionally, the choice to
   have DNS caching system should be left as an administrative
   decision.

 * Improved the HTTP standards compliance.  tinyproxy is still only
   HTTP 1.0 compliant, but it does respect the principal of "be
   liberal in what you accept, but strict in what you emit."

 * tinyproxy will now handle HTTP servers which do not properly close
   the connection after sending the HTTP message.

 * A new directive was added to limit the ports a CONNECT method is
   allowed to access.  This was needed to close a security problem
   whereby tinyproxy could be used a launching point for various
   attacks.

 * Fixed a problem where a socket was being closed twice.  This could
   lead to undefined errors.

 * The log rotation code has been moved out of the signal handler.
   This change was made necessary because the old code could cause a
   corruption in the log file when it was being rotated.

 * Fixed a bunch of memory leaks, and situations where memory was
   being freed twice (a potential security problem.)

 * Improved the list of headers which _must_ not be sent by the proxy
   to either the client or the remote server.

 * Fixed a problem where some log messages where not being written to
   the log file when tinyproxy was started.

 * Fixed all the error handling for the thread related functions.  The
   error codes were not checked correctly in the past.

 * Fixed a problem on some machines where threads were not being
   created correctly.  Also, add more logging code to indicate the
   successful creation of each thread.

 * Better error checking and reporting to both the administrator and
   the users.

 * Installation has been improved, as a copy of the tinyproxy.conf
   file is installed in the proper location.

 * tinyproxy send its information to remote servers in a more
   consistent manner.  This has fixed problems with sites like
   www.heise.de which did not work with tinyproxy 1.4.3.


 FUTURE
 ------

Here is a list of improvements slated for the next tinyproxy release:

 * Remove the HTML from within tinyproxy and move it into an external
   file.

 * Fix a "potential" blocking problem in the address lookup section.

 * Improve the security of the "Via" header to include support for
   data hiding.


 THANKS
 ------

The following people need to be thanked for all the work they've put
into improving tinyproxy since the 1.4.3 release.

 Hans-Georg Bork
 J.D. Bronson
 Tom Cross
 James Flemer
 Daniel Flemming
 Petr Lampa
 Jeffrey D. Wheelhouse
 Steven Young


 CONCLUSION
 ----------

tinyproxy is shaping up to be a very useful little tool.  There are
still long term goals relating to making tinyproxy a completely
HTTP/1.1 standards compliant proxy.  Anyone with ideas or patches for
tinyproxy should submit them to the tinyproxy-users mailing list at:

	tinyproxy-users@lists.sourceforge.net

or see the project's page at:

	http://tinyproxy.sourceforge.net/
        http://sourceforge.net/projects/tinyproxy/

Thanks again to all those who have helped to make tinyproxy a more
useful utility.

	Robert James Kaes
	May 9, 2002

Changes:
2002-05-08  Robert James Kaes  <rjkaes@flarenet.com>

	* configure.ac: The --with-config option now supports absolute
	paths.  If an absolute path is given, the full directory path and
	file name are extracted into their appropriate parts.

2002-05-07  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (connect_to_tunnel): Removed request logging code in
	the tunnel method since it breaks the "tunnel" concept.

2002-05-02  Robert James Kaes  <rjkaes@flarenet.com>

	* configure.ac: Added code to figure out exactly where to install
	the configuration file.

	* Makefile.am (install-data-local): Added a new install run to
	install the configuration file to the location specified in
	DEFAULT_CONF_FILE.

2002-04-28  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (process_client_headers):
	(process_server_headers): Added more error checking code and
	send HTTP errors back to the client to let them know what is
	happening.
	(handle_connection): If there was a server error when processing
	the headers, send an error back to the client.

2002-04-27  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c (thread_pool_create): Set the thread's status
	_before_ the thread is created.  Thanks to Hans-Georg Bork for
	solving this problem.  
	(thread_main): Fixed up the status setting code.  Removed
	duplicated code and explicitly set the status when needed.

	Fixed up all the code checking the return value from the pthread
	functions.  These functions return 0 if OK, but a _positive_ error
	code.

2002-04-26  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c: servers_waiting needs to be signed, since it can
	go below 0 without causing a problem, but if it wraps around all
	hell can break loose.

	* src/reqs.c (connect_to_tunnel): Changed the len type to an
	signed integer so that we can detect error conditions.  Thanks to
	Tom Cross for pointing out this problem.

	* src/log.c (send_stored_logs): Check the return value of
	hashmap_first() since it could be -1, indicating an empty hashmap.

	* src/reqs.c (process_client_headers):
	(process_server_headers): Test the return value of hashmap_first()
	since the hashmap could be empty (returning a -1 via hashmap_first.)

2002-04-25  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (remove_connection_headers): Handle both the
	Connection header _and_ the Proxy-Connection header.
	(free_request_struct): Check to see if request->path is actually
	allocated before trying to delete it.

	* src/hashmap.c: The hashmap will now handle multiple keys with
	the same value.  This change was need because some sites like
	Yahoo will send out multiple "Set-Cookie" lines.  The
	hashmap_keys() function has been removed and has been replaced
	with the iterator concept.  Also, a few of the functions have had
	either their arguments changed, or their return type.  Read the
	comments in the header file for more information.

2002-04-22  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c: Renamed the LOCKing macros and added debugging
	asserts to them.  Also, moved the mutex initialization into the
	thread_pool_create() function since I would like to use an error
	checking mutex, but my machine doesn't seem to work with it.  I
	left the code there in case I can get it to work later.

	* src/log.c (send_stored_logs): Added this function since the
	log_message() function will now stored the messages if the config
	file has not been processed yet.  This function is called from
	within main() to output the messages to the log file _after_ it's
	been initialized.

	* src/reqs.c (process_client_headers): Added the
	"Proxy-Connection" header to the list of client headers we don't
	forward to the remote machine.

	Added the the "Bind" directive.  This is used to indicate which IP
	address in a multi-homed machine you would like tinyproxy to bind
	out-going connections to.  This complements the "Listen" directive
	for incoming connections.

2002-04-18  Robert James Kaes  <rjkaes@flarenet.com>

	* src/sock.c (lookup_domain): Removed the call to inet_aton()
	since the gethostbyname() will handle the dotted-decimal case
	anyway.

	* src/reqs.c: Added a new show_stats field in the conn_s structure
	so that we will process the client's headers properly before
	trying to send a HTTP response back.

	* src/sock.c (getpeer_string): Removed the hstrerror() call since
	it's not supported on all machines, and it's not really needed
	anyway.

	* src/vector.c:
	* src/hashmap.c: Changed all calls to malloc, calloc, free, and
	strdup to the appropriate safe variety.
	(hashmap_delete): Fixed a memory leak where the maps were not
	being freed properly.

	* src/utils.c (debugging_strdup): Added this function to be used
	by the safestrdup() macro to replace all the calls to strdup().
	This should allow better tracking of the memory usage.
	Also, all the debugging_* functions have had asserts added to them
	to hopefully improve the quality of the code.

	* src/reqs.c (get_all_headers): Fixed a memory leak since I was
	not freeing the header variable, even though the hashmap makes a
	copy of it.  Thanks to Petr Lampa for finding this one.

	* src/tinyproxy.c (takesig): Moved the filter_destroy() code out
	of the signal handler and placed it inside of main().  Same
	reasoning as the rotate_log_files() changes below.

	* src/utils.c (rotate_log_files): Moved the log rotation code out
	of the signal handler and into it's own function.  Also improved
	the robustness of the code.  Credit to Petr Lampa for suggesting
	that system calls in a signal handler is bad magic.  Now the
	signal handler sets a flag which is responded to inside of
	thread_main_loop().

2002-04-17  Robert James Kaes  <rjkaes@flarenet.com>

	* configure.ac: Added test for pthread_cancel() since it doesn't
	seem to be available on every platform, even if the rest of
	pthread is.  Thanks to Daniel Flemming and Petr Lampa for
	reporting this problem.

	* src/sock.c (lookup_domain): Removed the LOOKUP_LOCK() and
	LOOKUP_UNLOCK() macros and replaced them with calls the LOCK() and
	UNLOCK().  The reason for this change is that I can not be sure
	that calls to gethostbyname() and gethostbyaddr() will not
	over-write the same static block of memory.  Potential problem
	pointed out by Petr Lampa.

	* src/reqs.c (handle_connection): If the initialize_conn()
	function fails, we need to close the socket.

	* src/acl.c (check_acl): Changed the code to accept the peer IP
	address and string address from the calling function (in this case
	handle_connection.)

	* src/thread.c (thread_main): Removed close(connfd) since it has
	already been closed from within handle_connection(). Thanks to
	Petr Lampa for spotting this.
	(thread_pool_create): Added test to make sure pthread_create()
	succeeds, and returns -1 if all the threads could not be created.
	(thread_main_loop): Added test to determine if the
	pthread_create() call succeeded.  Warns the admin if there was a
	problem, but tinyproxy will continue to run.

2002-04-15  Robert James Kaes  <rjkaes@flarenet.com>

	* src/anonymous.c (anonymous_insert): Now returns -1 upon error,
	and 0 upon success.  This simplified the code, since I don't care
	whether the success is from a successful insert, or because the
	string was already present in the hashmap.
	(anonymous_search): The function now returns a positive number
	greater than zero if the string was found, otherwise zero or a
	negative value is returned.  Needed to change the _one_ call to
	this function to use the new return method.

	* src/reqs.c (pull_client_data): Moved the common error exit
	together.  Better, smaller code is produced.
	(add_header_to_connection): Changed the code a bit to remove the
	call to strlen() and the variable associated with it.
	(establish_http_connection): Changed the three calls to
	write_message() and safe_write() into one combined call to
	write_message().
	(send_ssl_response): Replaced the three calls to safe_write() into
	one call to write_message().
	(extract_http_url): Moved the common error code into it's own
	section and jump to it upon error.

2002-04-14  Robert James Kaes  <rjkaes@flarenet.com>

	* src/utils.c (chomp): Added an assert to detect a design error.

	* src/reqs.c (read_request_line): Fixed the return type to be an
	unsigned variable, so that we preserve a negative return value
	upon an error.

	* src/sock.c (readline): Added a test for allocating enough memory
	for the whole_buffer.

	* src/conns.h: Replaced the error boolean with a pointer to an
	error string and error number for use in the
	send_http_error_message() function.

	* src/utils.c (indicate_http_error): Replaced the httperr()
	function with this one.  Instead of sending the error right away,
	we store the error string and number and send them _after_ the
	client headers have been processed.

2002-04-13  Robert James Kaes  <rjkaes@flarenet.com>

	* src/sock.c (opensock): If the Listen directive is in use, then
	we should bind outgoing address to this address.
	(listen_sock): Added error handling for the bind() and listen()
	calls when setting up the listening socket.

2002-04-12  Robert James Kaes  <rjkaes@flarenet.com>

	* src/scanner.l:
	* src/grammar.y: Added code for the ConnectPort directive.

	* src/reqs.c (add_connect_port_allowed):
	(check_allowed_connect_ports): Added these two functions (breaking
	by no new features rule) because they fix a security problem with
	tinyproxy.  These functions are needed to support the ConnectPort
	configuration directives, which allows the admin to specify which
	ports are allowed by a CONNECT method.  This is needed to prevent
	people from using tinyproxy to connect to mail servers (port 25)
	to do "bad things."

2002-04-11  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (remove_connection_headers): Moved all the code for
	detecting and removing headers based on the Connection header into
	this function.
	(get_content_length): Moved the check for a Content-Length value
	into it's own function.
	(process_client_headers): Changed the code to use the newer
	remove_connection_headers() and get_content_length() functions.
	(process_server_headers): Rewrote this function to handle the
	headers in a more consistent manner.  The connection header is
	handled correctly, and the content-length header is stored for
	later use.  Also, include a list of headers to not send.
	(relay_connection): Added a test to use the
	connptr->remote_content_length value if it's present.
	(write_via_header): Moved the creation/modification of the Via
	header into it's own function since it's used by both
	process_client_headers() and process_server_headers().  I still
	need to add code to allow you to "hide" the host information if
	you so choose.

	* src/conns.c: Added the remote_content_length variable to hold
	the number of bytes the remove server is _supposed_ to send to
	us.  This fixes a problem where the remote server doesn't close
	the connection after sending the body.  Problem was reported by
	James Flemer.

2002-04-10  Robert James Kaes  <rjkaes@flarenet.com>

	* src/Makefile.am: Removed the dependency on LIBOBJS and instead
	replaced it with an ADDITIONAL_OBJECTS variable defined within the
	configure script.  This allows greater control over what is
	compiled.

	* configure.ac: Removed the AC_FUNC_LSTAT macro since it wants to
	add lstat.o onto some machines.  I don't think the test was really
	needed anyway, so it's better to have it gone.

2002-04-09  Robert James Kaes  <rjkaes@flarenet.com>

	* src/anonymous.c: Changed this module to use the hashmap module
	instead of it's own internal linked list.  Common code is good. :)

	* src/reqs.c (process_client_headers): Fixed the XTINYPROXY_ENABLE
	section to actually check if xtinyproxy has been configured.
	(compare_headers): Removed this function since it's been replaced
	by a call to anonymous_search().

	* src/acl.c (make_netmask): James Flemer cleaned up the
	make_netmask() function to remove the big static table.  Much
	nicer now.

	* configure.ac: The configure script now doesn't include a check
	for the malloc.h header if tinyproxy is being compiled on an
	OpenBSD machine.  I might actually just remove the malloc.h header
	since the malloc/calloc/realloc/free functions are supposed to be
	in the stdlib.h header.  I also changed the LIBOBJS="" lines to
	AC_LIBOBJ() macro so conform to the new Autoconf rules.

	* src/hashmap.c:
	* src/vector.c: Cleaned up the include headers to _hopefully_ be
	more portable.  Who knows though.

2002-04-08  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c (thread_main_loop): Moved the looping code into
	this function from the main() function.  Just a bit of clean up.
	(thread_main): Changed the default cancellation point to
	asynchronous, which means immediately.  This should fix up the
	problem where the threads would not free correctly.

2002-04-07  Robert James Kaes  <rjkaes@flarenet.com>

	* src/hashmap.c:
	* src/vector.c: These two "modules" are used within the reqs.c
	file to better handle the headers from the clients.

	* src/reqs.c: Tonnes of functions changed.  The most significant
	is the process_client_header() function which is now using the
	hashmap functions to handle the Connection header more
	accurately.  With the addition of the add_header_to_connection(),
	get_all_headers(), and remove_connection_headers() function, the
	compare_header() function is _vastly_ simplified.  Just read the
	comments in the source itself for more information.

	* src/conns.c: Cleaned up the initialize_conn and destroy_conn
	functions to better handle error conditions.

2002-04-02  Robert James Kaes  <rjkaes@flarenet.com>

	* configure.ac: Patch from James Flamer which fixes a problem with
	the --disable-* options and also prevents -lc_r from being
	included on FreeBSD machines.

2002-01-13  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c (thread_kill_threads): Added a function to
	explicitly go through all the active threads and kill them.

2002-01-07  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c (thread_main): Added code to check if tinyproxy is
	being killed by the user.

2001-12-28  Robert James Kaes  <rjkaes@flarenet.com>

	* src/thread.c (thread_main): Added code to detect errors in the
	accept() function.

2001-12-23  Robert James Kaes  <rjkaes@flarenet.com>

	* src/utils.c (send_http_message): Use the write_message()
	function for creating and sending the headers to the client.
	(httperr): Use similar code to write_message() to create the body
	of the error message. I use this so that there is no limit to the
	size of the error message (better coding. :) I'm still trying to
	figure out how to combine this code with write_message() into a
	common function.

	* src/sock.c (write_message): Moved write_message() into sock.c
	since it's now accessed by more than just the reqs.c file.

	* src/tinyproxy.c (main): Made the error log for the idle time
	setting more verbose so that it's explains the reasoning
	better. Also, changed the level to WARNING.

	* src/reqs.c (process_client_headers): Cleaned up the code to send
	the Via header by using the write_message() function.

2001-12-22  Robert James Kaes  <rjkaes@flarenet.com>

	* src/tinyproxy.h: Add a test for the MSG_NOSIGNAL define, so it's
	apparently a non-standard extension to send(). Also, moved the
	<sys/resource.h> include after the includes for the various time
	include files.

2001-12-19  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (write_message): Encapsulate code to handle sending
	snprintf() built lines to a file descriptor.
	(process_server_headers): Removed duplicate code and used the
	return value from readline() instead of recalculating it by
	strlen().

2001-12-18  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (connect_to_tunnel): Moved the tunnel connection code
	into its own function.
	(TUNNEL_CONFIGURED): Added a macro to help simplify the tests for
	tunneling support.

2001-12-17  Robert James Kaes  <rjkaes@flarenet.com>

	* src/reqs.c (handle_connection): Also log the request when
	tinyproxy is in tunneling mode. This was implemented by
	Hans-Georg Bork.

2001-12-16  Robert James Kaes  <rjkaes@flarenet.com>

	* src/sock.c (safe_write): Add assert() calls to make sure the
	arguments are sane.

	* src/reqs.c (read_request_line): Ignore any blank lines if
	tinyproxy is expecting a request line. This is suggested by
	RFC2616 for a tolerant application.
	(UPSTREAM_CONFIGURED): Added a macro to encapsulate the test for
	upstream proxy support.
	(establish_http_connection): Lines are now composed in a buffer
	and then sent to the safe_write() function. This was done because
	some sites (like www.heise.de) do not like having the request
	line sent in pieces.

2001-12-15  Robert James Kaes  <rjkaes@flarenet.com>

	* src/sock.c (safe_write): In addition to handling "interruption"
	errors, continue sending the buffer until it has been fully sent
	or a fatal error occurred.

	* src/anonymous.c: Removed the calls to the ternary tree and just
	use a basic linked list. The ternary tree might have been slightly
	faster, but since tinyproxy is only dealing with a "few" anonymous
	headers the ternary tree code was _way_ overkill.

	* src/sock.c (lookup_domain): Removed the DNS cache since
	according to RFC2616 a HTTP client should not cache an address if
	the client doesn't also respect the TTL value. tinyproxy does not
	have a complete DNS resolver. Additionally, a caching DNS system
	is an administrator decision which should not be included in
	tinyproxy.

2001-11-25  Robert James Kaes  <rjkaes@flarenet.com>

	* src/buffer.c (makenewline): This function now copies the actual
	data itself. This means any malloc() and free() functions are
	closer together, instead of malloc()ing the data and then not
	free()ing it.
	(add_to_buffer): Add a new "line" to the buffer. I'm trying to
	have more of the data go through the buffering code. Basically the
	safe_write() function should only be called when the data needs to
	be sent _now_.
	(read_buffer): Fixed the code so that buffering _actually_
	happens. :) The code will now buffer up to MAXBUFFSIZE bytes
	(which is currently 48 KB.)

2001-11-24  Robert James Kaes  <rjkaes@flarenet.com>

	* configure.ac: Changed the extension of the configure.in to match
	the new standard for the autoconf program. Also, replaced all the
	obsolete macros with the new standard. autoconf 2.52 is now the
	minimum required to process this file.

2001-11-23  Robert James Kaes  <rjkaes@flarenet.com>

	* src/sock.c (readline): Limit the maximum size of the buffer so
	that we don't create a situation where tinyproxy could use all the
	memory in a system. Right now we're limiting a line to a maximum
	of 128KB.