Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

Migrating Trac from Hosted Apps

Note: *The Hosted Apps platform was retired on 2014-06-19.

Not Recommended: migrating Trac from Hosted Apps to Project Web

The SourceForge team has evaluated this issue and has determined that the Project web platform is not a viable option a Trac installation.

The rest of this document will remain for users that still want to try this regardless, however SourceForge will not assist with installation or maintenance tasks in accordance with our third party application policy for Project web.

Status

BROKEN as of 2013-12-24

1) Get backup of your Trac database

You will find a backup of your Trac data in your project's (or user's) web directory root. For a project named PROJECT these are:

/home/project-web/PROJECT/p_PROJECT_trac.sql.gz (the database dump)
/home/project-web/PROJECT/p_PROJECT_trac.tar.gz (the web files)

For a user you will have /home/user-web/USER/u_USER_* files instead. You can copy these using rsync/scp/sftp from web.sourceforge.net using your sourceforge username and password.

2) Enable mysql for your project

In classic:

Go to project home page e.g. http://sourceforge.net/projects/paintown

Select Project Admin => Features

Feature Settings page appears with Enabled Features table

In Project Database (MySQL) row, click Manage

MySQL Service page appears. This shows information including

Host name e.g. mysql-p
Database name prefix e.g. p261620_
Database users e.g. p261620ro, p261620rw, p261620admin
In what follows, replace the above with your own host name, name prefix or users

Enter passwords for the three MySQL user accounts, if you have not already done so

Click Set passwords

A green message will appear, "Password data recorded. Changed passwords will be available on the MySQL server in approximately five minutes."

In Allura, these same tools are available at.
Admin -> Tools -> Mysql

Make sure that you’ve set passwords for your various access accounts, and make note of your required database hostname and prefix for the following step.

3) Copy files and get access to shell:

Secure copy files to your SourceForge account

  $ scp paintown_trac_mysqldump.sql sfuser@web.sourceforge.net:
  sfuser@web.sourceforge.net's password:
  paintown_trac_m 100% |*****************************| 2610 KB 00:01

  $ scp paintown_trac_files.tar sfuser@web.sourceforge.net:
  sfuser@web.sourceforge.net's password:
  paintown_trac_f 100% |*****************************| 8540 KB 00:04

Log into the SourceForge secure shell

  $ ssh -t sfuser,paintown@shell.sourceforge.net create
  sfuser,paintown@shell.sourceforge.net's password:

  Requesting a new shell for "sfuser" and waiting for it to start.
  queued... creating... starting...

  This is an interactive shell created for user sfuser,ogsa-dai.
  Use the "timeleft" command to see how much time remains before shutdown.
  Use the "shutdown" command to destroy the shell before the time limit.
  For path information and login help, type "sf-help". 
  $

4) Create database:

Create a database for Trac

Remember to replace mysql-p, p261620admin, p261620_ with your database prefix, database administrator user and database prefix as described in step 2

  $ mysqladmin -h mysql-p -u p261620admin -p create p261620_trac
  Enter password:

Set UTF-8 encoding to created database

  $ echo "ALTER DATABASE p261620_trac DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;" | mysql -h mysql-p -u p261620admin -p p261620_trac
  Enter password:

Load the database dump

  $ mysql -h mysql-p -u p261620admin -p p261620_trac < ~/paintown_trac_mysqldump.sql
  Enter password:

(Alternately, you may use the web interface at https://mysql-t.sourceforge.net. Select the _trac database, select the "import" tab, upload the paintown_trac_mysqldump.sql and import it.)

Clean up

  $ rm ~/paintown_trac_mysqldump.sql

5) Install some Python prerequisites

Install virtualenv:

 $ cd ~
 $ wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.8.2.tar.gz
 $ tar -vzxf virtualenv-1.8.2.tar.gz

 $ mv virtualenv-1.8.2/virtualenv.py . 
 $ rm -r virtualenv-1.8.2
 $ rm virtualenv-1.8.2.tar.gz
 $ python2.6 virtualenv.py --system-site-packages /home/project-web/paintown/venv

New python executable in /home/project-web/paintown/venv/bin/python
Installing setuptools..............................done.
Installing pip.....................done.

Be sure to verify that you're installing the latest version of virtualen.

See Installing Trac without Root Privs for more detail on why you're doing this.

Install other prerequisites:

/home/project-web/paintown/venv/bin/easy_install Pygments

6) Now install Trac

Go to your CGI directory, and install Trac with easy_install

cd /home/project-web/paintown/cgi-bin
/home/project-web/paintown/venv/bin/pip install trac

7) Create your Trac environment

cd /home/project-web/paintown/
tar -xvf ~/paintown_trac_files.tar
mv paintown trac_env

Remove invalid link

rm trac_env/htdocs/ha-css

Clean up

rm ~/paintown_trac_mysqldump.sql

Modify /home/project-web/PROJECTNAME/trac_env/conf/trac.ini and change database connection string. Also see if there's anything in there that you want to tweak.

[trac]
database = mysql://p261620admin:PASSWORD@mysql-p:3306/p261620_trac

Be sure to replace the bolded variables with your project's mysql information from steps above.

8) Deploy the CGI script

Now, deploy the cgi script.

/home/project-web/paintown/venv/bin/trac-admin /home/project-web/paintown/trac_env deploy /home/project-web/paintown/
chmod +x /home/project-web/paintown/cgi-bin/trac.*

This step may fail due to missing mysql bindings. You can install these bindings via the process described at https://sourceforge.net/p/forge/community-docs/Migrating%20Trac%20from%20Hosted%20Apps/?limit=25&page=3#71f3 until such time as we have these bindings installed.

Trac should now be available at http://paintown.sourceforge.net/cgi-bin/trac.cgi


Related

Community Docs: Hosted Apps Retirement

1 2 3 .. 5 > >> (Page 1 of 5)
  • Mike Jackson
    Mike Jackson
    2012-07-03

    I tried these on http://sourceforge.net/projects/ogsa-dai and all was fine to step 6. For step 6:


    $ cd /home/project-web/ogsa-dai/cgi-bin
    $ ~/bin/python setup.py build_py install
    /home/users/m/mi/mikej888/bin/python: can't open file 'setup.py': [Errno 2] No such file or directory

    So I tried:

    $ cd /home/project-web/ogsa-dai/htdocs/trac
    $ ~/bin/python setup.py build_py install
    ...
    Finished processing dependencies for Trac==0.12.3

    Which seems OK.

    Will there be more info coming as to how to add the backed-up files, configure Trac to point at MySQL and to start the server?

     
  • Rich Bowen
    Rich Bowen
    2012-07-03

    Mike,

    Yes, this is a work in progress, and I should have marked it as such more clearly. There are still several steps remaining.

    I believe you must install it in the cgi-bin directory, and I might have mixed up some paths somewhere along the way. I'll be working on this more today.

     
  • Mike Jackson
    Mike Jackson
    2012-07-03

    Hi Rich,
    Ok, cool, I'll try again when you're done.

     
  • Jools Wills
    Jools Wills
    2012-07-03

    Stage 7 fails due to missing mysql bindings for python or some other problem :

    Creating and Initializing Project
    Initenv for '/home/project-web/xbmc4xbox/trac_env' failed. 
    Failed to create environment.
    Cannot load Python bindings for MySQL
    Traceback (most recent call last):
      File "build/bdist.linux-x86_64/egg/trac/admin/console.py", line 437, in do_initenv
      File "build/bdist.linux-x86_64/egg/trac/env.py", line 214, in __init__
      File "build/bdist.linux-x86_64/egg/trac/env.py", line 402, in create
      File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 145, in init_db
      File "build/bdist.linux-x86_64/egg/trac/db/api.py", line 195, in get_connector
    TracError: Cannot load Python bindings for MySQL
    Initenv for '/home/project-web/xbmc4xbox/trac_env' failed.
    

    its also possible i made a mistake so ill double check.

     
    Last edit: Jools Wills 2012-07-03
  • Rich Bowen
    Rich Bowen
    2012-07-03

    Yes, that's exactly the problem I'm trying to solve myself. If you have any insight into this at all, please be sure to let me know. Meanwhile, I'm working on it.

     
  • Jools Wills
    Jools Wills
    2012-07-03

    easy_install MySQL-python should work but that would need the headers on the machine - perhaps that can be added to the hosting environment. dont know how to build them against headers installed in homedir etc.

     
  • defnax
    defnax
    2012-07-03

    i get the same problem

     
  • Joe
    Joe
    2012-07-09

    Mysql connector can be copied from system python instance - updated.
    Created database has to have UTF-8 encoding - updated.

    Users home dir seems to be not accessible from web-server. virtualEnv should be created in web-server visible directory - needs update.

    Load of backup should be done after initenv, otherwise the initenv will fail - needs update.

     
  • Joe
    Joe
    2012-07-09

    Updated wiki to install virtualenv into web folder.
    Removed unnecessary step of downloading and extracting Trac (done by easy_install in next step)
    Updated to use pip instead of easy_install to automatically installs dependencies.

    Using current instructions I'm able to get clean Trac running. However loading database dump breaks the installation, because database needs to be updated and the upgrade command fails on

    $ /home/project-web/***/venv/bin/trac-admin /home/project-web/***/trac_env upgrade
    OperationalError: (1050, "Table 'cache' already exists")
    

    Database inconsistencies needs to be solved.

     
    Last edit: Joe 2012-07-09
  • Joe
    Joe
    2012-07-09

    Database problem solved. Current instructions should import both database and files.

    The last problem is lack of Subversion binding for Python. Trac is working but connection to Subversion repository is not.

     
    • Chris Tsai
      Chris Tsai
      2012-07-09

      Thank you for your contributions, it is greatly appreciated.

      Not being able to connect to the repo is expected from the Project web space, but I'll bring this to our team to see if this is something we can address or not.

       
  • Hi Joe,

    First of all: nice and very helpful tutorial. Will there be support for the SourceForge users or do I have to create new users manually?

     
    • Chris Tsai
      Chris Tsai
      2012-07-09

      Hello Matthias,

      First of all, Joe is not SourceForge staff, he's just a user making some greatly appreciated contributions.

      Second, we do have plans to get OpenID authentication set-up for our site (in-progress, to be completed this month), once that's in place, we'll work on updating these docs with how to implement that the various apps. That way, you should still be able to let folks login using their SF accounts.

       
  • I'm very confused by all this project upgrading and dropping of existing functionality.

    I do not want to have unsupported/custom features enabled in my SF projects. So if you are going to dump TRAC, I want to dump it too and migrate to alternative products. I do not want to go through a complicated data migration process and a custom MySQL / Python / shell web server setup. Instead I want to manually migrate existing information on Trac to alternative products that have long-term support.

    TRAC provides basically two functions: Wiki and Tracker.

    I know that SF has a tracker feature. I can use this one for the Tracker part. Ok so far.

    But What does SF offer for Wiki? Apparently MediaWiki is being dropped as well so that is not an option.

    When I click on "available features" for my projects I only see a very limited set of features: the mentioned SF tracker, some source control products and Forum. But no Wiki. Do you support a Wiki at all?

     
    • Chris Tsai
      Chris Tsai
      2012-07-09

      Yes, we do have a wiki, in fact, you just commented on it. This is part of the new SourceForge system the team has been working on for the past few years. Your project however sounds like it's still on the classic platform (which is no longer being developed). To upgrade to the new system, go to Project Admin -> Settings, then on the right side, "Opt-in Upgrade".

       
      • That page mentions "Trac will no longer integrate with SVN"

        When migrating Trac to the project web, will that Trac installation integrate with SVN ?

         
    • Jools Wills
      Jools Wills
      2012-07-10

      I agree and now have moved, migrated my data to my own host. sorry to have to do it, but there was no choice. sf was too slow. and this just made more trouble..

       
  • Jools Wills
    Jools Wills
    2012-07-10

    In case it is useful to someone else - I migrated all trac contents to a redmine install (not hosted at sourceforge). this was done using redmine built in trac importer which worked perfectly. Unfortunately I decided not to stay with sf, due to performance problems of the platform.

     
  • defnax
    defnax
    2012-07-10

    i get some trouble here:

    -bash-3.2$ /home/project-web/retroshare/venv/bin/trac-admin /home/project-web/retroshare/trac_env deploy /home/project-web/retroshare/ chmod +x /home/project-web/retroshare/cgi-bin/trac.*
    Error: Invalid arguments

    deploy <directory>

    Extract static resources from Trac and all plugins

    trac not works yet.

     
    • Joe
      Joe
      2012-07-10

      Those are two commands, not just one.

      1. /home/project-web/retroshare/venv/bin/trac-admin /home/project-web/retroshare/trac_env deploy /home/project-web/retroshare/
      2. chmod +x /home/project-web/retroshare/cgi-bin/trac.*
      

      Maybe there should be $ in front of each command, but that would disallow to copy/paste more commands at once.

       
  • defnax
    defnax
    2012-07-10

    thanks but now i get errors with upgrade...

    -bash-3.2$ /home/project-web/retroshare/venv/bin/trac-admin /home/project-web/retroshare/trac_env upgrade
    ProgrammingError: (1146, "Table 'r**_trac.revision' doesn't exist")

     
    Last edit: defnax 2012-07-10
  • Spencer Oliver
    Spencer Oliver
    2012-07-10

    ok i have created a project trac instance on the web space - one word it is slooooooow.
    This really needs looking into otherwise migration is not worth it.

     
  • Jon Rafkind
    Jon Rafkind
    2012-07-10

    Spencer, is it much slower than the hosted app version? How slow would you say the web space version is?

     
    • Joe
      Joe
      2012-07-10

      Speed issues are expected I believe. I think caches are turned off by default and also cgi interface is not the most performant one - wsgi or at least fcgi would probably be faster. Now on each request the whole interpreter it started and initialization is done again and again.

       
      • Chris Tsai
        Chris Tsai
        2012-07-10

        Thank you for the speed reports. We'll assess this ourselves once we're at the "Updates to Project web to support apps" stage in our timeline. We want to make sure that this works well for users, but will need to evaluate our options.


        Note: there's a bug with some comment per page limits and pagination not working with wikis. The team is looking into a quick fix for this page, then they'll be working on a better, long-term fix. Until this is fixed, new comments on this page, will not be visible. I'll still get an email about it, so if appropriate, I'll try to follow-up as best I can.

        Issue has been documented here: http://sourceforge.net/p/allura/tickets/4525/

         

        Related

        Community Docs: Hosted Apps Retirement


        Last edit: Chris Tsai 2012-07-11
1 2 3 .. 5 > >> (Page 1 of 5)