SourceForge.net

SourceForge.net staff are constantly working to improve
the quality of our service offering. In recent months,
a number of large enhancements have been made to the
SourceForge.net site and developer services, as
documented at: https://sourceforge.net/docs/A03/

In coming weeks, SourceForge.net staff will be
performing the following changes to the project shell,
database and web services to improve maintainability
and the performance of our service offering. We
recognize that a number of these changes will require
projects to reconfigure their web applications and
modify their web content; we have merged all of these
changes in to a single roll-out as that no further
changes should be needed any time soon.

The time line for this change set will be established
by the SourceForge.net team; this time line will be
posted as follow-up comments to this master RFE.
Instructions for submitting questions about these
changes may be found at the bottom of this notice.

Documentation:
New Site Documentation (https://sourceforge.net/docs/)
will be released regarding the project shell, database
and web services. This new site documentation replaces
all of our existing documents on these subjects,
providing a more consistent set of documentation that
includes details of all recent service changes.

Logo Display:
The URL for SourceForge.net logo display is changing.
The sourceforge.net hostname should be replaced with
sflogo.sourceforge.net in all SourceForge.net logo
display URLs. Instructions have been updated
accordingly. These changes allow us to more rapidly
execute changes to the logo service and segment
unrelated traffic. Logo display continues to be
mandatory for all projects that use our project web
servers.

Cron service:
Cron service is being re-enabled on the project shell
servers. All cron jobs will need to be re-established.
Backup copies of prior cron jobs will be available for
review. It is mandatory that crontab files include a
description of what a cron job is supposed to be doing;
this description will be used by SourceForge.net staff
when troubleshooting performance issues. Cron jobs
should be configured to run no more frequently than
once per ten minutes. Instructions for creating a
crontab (used to schedule cron jobs) are provided in
our new Site Documentation. Cron service was
previously disabled due to widespread misconfiguration
and abuse; this change re-enables this valuable service
and establishes policies to help keep things maintainable.

Outbound email from project web and shell servers:
Outbound email from project web servers will be
blocked. Outbound email from the project shell servers
will be permitted. Projects which use project web
applications that require outbound email will need to
modify these applications to queue the mail or mail
operations to their project database. Projects may
then perform the sending of mail via a cron job on the
shell server. All mail must originate from the user
whose account is being used for the cron job. Projects
may create a utility account, to be maintained by
project administrators, if needed for this purpose.
Outbound mail from the shell server is being segmented
from other site mail to reduce the chances of impact to
site mailings due to project-caused spam listings.
These changes are being made to reduce abuse from the
project web servers, including the generation of spam
by third parties.

Transition to SSH key authentication:
Password authentication to the project shell servers
will be deprecated in early 2005. All users are
encouraged to establish SSH keys for authentication, as
described in the Site Documentation. This change is
being made to improve security and reduce password
dependency; both SSH1 and SSH2 keys are supported. We
have provided support for SSH key authentication for
several years.

Major download server upgrade:
We have recently completed a large upgrade of our
primary download server from 1TB of disk storage to
3TB. This upgrade has allowed us to leverage the
project download server network in new ways.

Sandbox and snapshot file services:
We have begun the roll-out of sandbox and snapshot
services to projects. Our sandbox service permits the
maintenance of a directory structure on the download
servers via rsync, allowing projects to maintain the
directory structures needed for APT repositories and
similar. Our snapshot service allows the rapid posting
of nightly snapshots to the File Release System,
improving download performance. Automated tools are
provided for maintaining project snapshots. These new
services have been designed to meet the needs of the
bulk of the projects that currently use project web
service to serve files; further improvements to the
File Release System on the SourceForge.net site are
planned and under development.

Blocking of large file transmission:
We previously implemented throttling of large files
served via project web. In the near future, we will
begin blocking the transmission (via 403 error) of
large files via project web. All projects that need to
serve large files via project web should instead
request that we enable sandbox or snapshot service for
their project. This change is being made to leverage
our new download server capacity and reduce the
performance impact of large file serving on the project
web servers.

Backups and monitoring:
We have used this upgrade as an opportunity to initiate
improvements to our monitoring and backups; both
fundamental tools we use to ensure high levels of
service quality. These changes should be transparent
to the user. As always, our backups are used for
recovery in the event of catastrophic system failure.
We do not provide on-demand file recovery for projects
and projects are encouraged to maintain their own
backups as described at:
https://sourceforge.net/docman/display_doc.php?docid=6840&group_id=1

MySQL database version upgrade, centralized phpMyAdmin:
MySQL database services will be upgraded to MySQL
4.0.x. At that time, we will begin offering a
centralized phpMyAdmin service, allowing projects to
more readily maintain their project database. An
overview of changes in MySQL 4.0.x may be seen at:
http://dev.mysql.com/doc/mysql/en/News-4.0.x.html
http://dev.mysql.com/doc/mysql/en/Upgrading-from-3.23.html
This change is being made to improve the quality of our
MySQL database service; this is a standard version
upgrade to the next stable release of the MySQL
database software.

MySQL query runtime cap:
Previously-implemented caps on MySQL query runtime (to
180 seconds) will be lowered to 120 seconds. This
change is being made to further reduce the impact of
poorly-optimized queries and inappropriate queries.

Shell server OS upgrade:
The Operating System load on the project shell server
will be upgraded to match the current load on the
project web servers. Package load will not be matched,
but CGI scripting languages will be loaded, to permit
cron jobs to be created in the same language as the
application. We will continue to offer editing tools
on the shell server, to permit the generation and
maintenance of project web content; this continues to
be the primary purpose of the project shell server.
The Compile Farm host which matches this configuration
(x86 Fedora) will be updated to have a package listing
that more directly matches the packages included on the
project web servers. All application development work
should occur using the Compile Farm rather than the
shell server. This change is being made to provide a
more consistent, useful environment to maintain your
project web site.

Group directory is read-only on project web:
All existing group directory space will be mounted
read-only. A new shared temp directory space will be
created under /tmp/persistent on the project shell
servers and project web servers. Projects should
create a project- specific subdirectory to hold their
data. Regular backups of all project data placed
within this space should be made. We strongly
encourage all projects to transition from their
file-based applications to using the MySQL database
service we offer as the data backend. Group directory
space will continue to be read-write on the project
shell servers, to permit maintenance of project web
content. This change is being made to discourage the
use of file-writing applications in lieu of our MySQL
database service and to combat several classes of
malicious applications.

Change to MySQL database server host name:
All MySQL database access should performed against the
MySQL server whose letter matches the first letter of
your project UNIX name. Previously, projects would
access 'mysql.sourceforge.net'. If your project UNIX
name is 'projectname', you would now access
'mysql-p.sourceforge.net'. This change is being made
to permit increases in the number of MySQL database
servers used to serve project traffic.

Custom Project VHOSTs (Virtual Hosts) cap:
The number of custom project VHOSTs permitted per
project will be capped at ten. A very small number of
projects have been found to use more than ten custom
VHOSTs; we have found that most cases where this has
been done in the past relate to abuse of the project
web service for display of inappropriate content or
commercial gain. This change is being made to reduce
abuse and improve the maintainability of our custom
VHOST service.

Scanning for Vulnerable Applications:
SourceForge.net staff will begin scanning for
vulnerable versions of applications which have been
loaded in project web space. Project administrators
will be notified of our findings. All projects are
responsible for performing regular upgrades to the
applications they load in to project web space. This
new activity will be performed periodically to reduce
the impact of malicious code.

Quotas:
All projects are restricted to 100MB of space in their
group directory structure on the shell and web servers.
We will begin sending automated notices of overages to
project administrators. In the past, notices were sent
on a manual basis. With the new sandbox and snapshot
tools available, and the other changes described here,
the need of projects to exceed the standard quota
should be reduced. We will continue to consider
extended quotas for projects that have special needs.


This substantial listing of changes represents the
first large set of changes made to the project web
service in more than two years. While we have
previously made changes to the equipment and software
load, these functional and policy changes will
significantly help to improve the reliability and
maintainability of our project web service.

QUESTIONS, CONCERNS, SUPPORT:
Projects are encouraged to let us know directly if they
have questions or concerns regarding this
implementation. Inquiries should be directed to
SourceForge.net staff via our Confidential Support
queue at:
https://sourceforge.net/tracker/?func=add&group_id=1&atid=596964

Pending the completion of these new limitations and
service enhancements, we will begin implementation
planning for an additional set of service enhancements,
focusing on providing brand new services related to our
project shell, database and web services.

We appreciate your patience and assistance during this
transition of the project web services.

Thank you,

SourceForge.net staff