NCCOOS Trac Projects: Top | Web | Platforms | Processing | Viz | Sprints | Sandbox | (Wind)

Sprint Goals

  1. Migrate old SVN projects to new SVN instance
  2. Develop framework to organize existing code
  3. How to use wiki to reference/shortcut/organize/manage SVN

Sprint Tasks

...to meet above goals

  • Overview of wiki formatting
  • Overview/best practices of SVN
  • List available code
  • Decide on components and projects to organize above code
  • start placing code in SVN
  • Start wiki pages to point to code of interest

Available Projects

Top Level project tree

Projects give control over areas of work and responsible people. Each project has a number of components which are deployable source trees within a given project. The following outline is a brain dump of current code pieces that are maintained and stored by Sara, Jesse, and Chris to see how they fit into the framework of Projects. The "production" processes live and run on various computers.

Some possible projects are:

  • Platforms
    • Sara
      • data_acquisition (eVB)
      • daqLib (eVC)
      • data_transfer (eVB)
      • commLib (eVC)
  • Telemetry
  • Repository
  • Data Management change this project name to Data Processing
    • Jesse
      • allinsitu_2_shape (NCCOOS)
      • GDP drifters (via AOML) to Xenia DB (seacoos)
      • CODAR to shapefile (seacoos)
      • QuikSCAT (seacoos)
    • Sara (buoy, tower) (met, ctd, adcp)
      • populate level0 (realtime and manual) (push and get)
      • raw2proc (level0 to level1)
      • proc2ndbc (push fm13 and fm64 to NDBC)
      • proc2latest (provide SEACOOS netCDF of latest data)
      • generate html tables (realtime)
    • Sara (codar) (surface currents)
      • tar2ms.pl (tar monthly CODAR raw spectra and rangefiles and and scp to mass-storage)
      • combine radial sites to total surface currents
      • radials proc2latest (provide SEACOOS netCDF of latest data)
      • totals proc2latest (provide SEACOOS netCDF of latest data)
  • Database
    • Jesse
      • nccoos_obs to DB (netCDF of CODAR totals & JPOP to NCCOOS_OBS DB)

  • Visualization
    • Jesse
      • Chameleon application (NCCOOS)
      • NCCOOS interactive map application (NCCOOS)
      • MS Mapfiles (NCCOOS and SEACOOS)
      • OGC CGI wrappers (NCCOOS and SEACOOS)
      • OGC mapfiles - parallel to SC feeds (NCCOOS and SEACOOS)
      • Sandbox WMS applications (SEACOOS)
    • Sara
      • proc2tsplot (last 24 hours, last 7 days, last 30 days timeseries plots)
      • lastest2map (latest obs and various model maps)

SVN and Trac Demonstration

A demostration of how to implement our framework of Projects and code components in SVN and Trac. We will be creating the Platforms repository and project and component daqLib under the Platforms.

Keep in mind that a Trac Project corresponds to an SVN Repository and a Trac Component corresponds to an SVN folder in Repository

Creating Subversion Repository

login as root to coriolis.marine.unc.edu 

% ssh root@coriolios.marine.unc.edu
passwd:

% cd /var/srv
% ls
svn trac

View instances of svn on coriolis 

% ls svn
nccoos seacoos

Use svnadmin to create and manage repository. Get a list of help commands and help on specific command usage 

% svnadmin help
% svnadmin help create

list repositories of nccoos 

% ls svn/nccoos
datamgmt portal scratch

Create the SVN repository. NOTE: specify --fs-type fsfs because documentation is incorrect. It is fsfs is not the default. 

% svnadmin create --fs-type fsfs /var/srv/svn/nccoos/platforms

Verify newly created SVN repository 

% ls svn/nccoos
datamgmt platforms portal scratch
% ls snv/nccoos/platforms
conf dav db format hooks locks README.txt

Change permissions on SVN repository 

% cd /var/srv/svn/nccoos
% chown -R apache:root platforms

Add/Set permissions in dot-access of SVN 

% cd /var/srv/svn/nccoos
% ls -a
. .. .access datamgmt platforms portal scratch
% vi .access

insert line under [groups] who will work on this code 

platform-developers = haines

insert line under [scratch:/] 

@platforms-develops = rw"

Allow all developers to create new components 

[portal:/] as [platforms:/]
@platforms-developers = rw
cbc = rw
* = r

If want to refine who works on a specific component you can be very specific in .access, for example 

[platforms:/daqLib]
cbc = rw
haines = rw
* = r

Now we are ready to create folders and submit code to SVN. See the section on committing code to SVN and creating component folders if given permission to do so. But first we need to set up the Trac project and component.

Create Trac Project and Component

Use initenv command trac-admin to create your project. Answer interactive questions. You will need to know exact path to svn repository that you just created. 

% trac-admin /var/srv/trac/nccoos/platforms initenv

Project Name [My Project]: NCCOOS Observing Platforms
Database connection string [sqlite:db/trac.db]> use default
Path to repository [/srv/svn]> /var/srv/svn/nccoos/platforms
Templates directory [/usr/share/trac/templates]> use default
<...more stuff...>
Congratulations!

Verify the created component directory for trac just like with svn 

% ls trac/nccoos
datamgmt platforms portal scratch

Configure Trac to recognize the new project 

% cd /var/srv/trac/nccoos/platforms/conf
% ls
trac.ini
% vi trac.ini
  • remove all the default stuff
  • insert the following file that is a desired configuration from nccoos/portals project
  • insert ../../portal/conf/trac.ini
  • edit/change all "portal" stuff that was copied to apply for "platforms"

(Note: we need an example or template here)

Copy existing htdocs from portals for logo and facoicon.ico 

% cd ../htdocs
% cp -a ../../portal/htdocs/* .

Change permissions recursively on the Trac project directory and files 

% cd /var/srv/trac/nccoos
% chown -R apache:root platforms

Get the Admin Menu on Trac Webpage for project using trac-admin interactively 

% cd /var/srv/trac/nccoos/platforms
% trac-admin /var/srv/trac/nccoos/platforms/

List permissions and get help within Trac 

Trac> permission list
Trac> help 
< ... >
permission add <user> <persmission>

Trac> permission add cbc TRAC_ADMIN
Trac> exit

Apache Server

You will need to make some changes to the Apache server configuration and restart the http daemon.

Edit NCCOOS configuration 

% cd /etc/httpd/conf.d
% vi nccoos.conf

duplicate <Location /portal> and change 3 occurences of portals to platforms

Restart Apache now that you have changed a configuration 

% service httpd restart
Stopping httpd
Starting httpd ...

Trac Site Admin

User cbc can now use web-based admin tools from the Admin menu on the Trac for the Platforms Project

Via the new trac site visit the Admin menu -> General -> Permissions to immediately restrict the following permissions to "authenticated" users:

  • TICKET_CREATE
  • TICKET_MODIFY
  • WIKI_CREATE
  • WIKI_MODIFY

You might also be grant the following permission to "authenticated", based on how you think your Trac site might be used by project members:

  • WIKI_DELETE
  • TICKET_APPEND
  • TICKET_CHGPROP
  • REPORT_CREATE
  • REPORT_DELETE
  • REPORT_MODIFY
  • MILESTONE_CREATE
  • MILESTONE_DELETE
  • MILESTONE_MODIFY

Committing Code to Subversion

http://subversion.tigris.org/ Tortoise

SVN Book http://svnbook.red-bean.com/ pg 21 Subversion Architecture Diagram

Lock-Modify-Unlock Model can do this in SVN, but it is not cool. Copy-Modify-Merge Model is the way SVN operates and is optimistic.

Components Usually Have:

  • Trunk - main code housed here
  • Branches - under development/ not in production
  • Tags - ID a release level of revisions in trunk


Subversion on Windows: Tortoise SVN

Create a Component

  • Right Click to get Explorer Menu -> TortoiseSVN -> repo-browser
  • Navigate to project of interest - right click
    • Create folder... - component name
      • navigate to new component - right click...
        • Create folder... - "trunk"
          • Create folder... - component name
        • Create folder... - "branches"
        • Create folder... - "tags"

Import Code to Subversion

  • Sub folders ready? Right click on local folder containing code to version control
  • Choose Import in TortoiseSVN sub-menu
    • typical URL: http://svn.instance.org/repository/component (e.g http://svn.nccoos.org/scratch/daqlib)
    • Best practice = use trunk framework
      • like http://svn.instance.org/repository/component/ trunk /component
    • authenticate: UID and PW same as trac
  • Files are then added to SVN
    • Version numbers increase on an entire repository basis
    • Adding new component will increase the entire repository version number

Create a Working Copy - SVN Checkout

  • Right click (anywhere outside of a toolbar or application) and choose SVN Checkout from Explorer Menu
  • Select the URL of the repository of interest
  • Pick/Create a local Checkout Directory to create a Working Copy
  • Choose the Head Revision or a Revision Number
  • Folder is created: green check mark = folder has not changed from Checkout
  • A .svn folder is also created on Checkout to hold Subversion bookkeeping - don't mess with this folder!
  • Ready to make changes to code

Commit Changes Back to the Repository

  • Can commit single changed file or entire container folder
  • Right click and select SVN Update to check that files have not been changed since initial Checkout
    • If files have changed
      • Merge: other changes do not overlap your changes)
      • Resolve Conflicts: other changes do overlap your changes
  • No external changes - right click on file or folder and select SVN Commit
  • Add comments to describe the changes being committed

Export Code to Deploy

  • Right Click -> Explorer Menu -> TortoiseSVN -> Export - like Checkout but with no bookkeeping

Remove an Entire Component

  • Explorer Menu -> TortoiseSVN -> repo-browser - can remove an entire Component
    • Removal actually creates a new version
  • Can export a previous version to access removed component code

Working With Tags & Branches

  • Using a working copy with trunk folder at least checked out
  • Right Click on trunk folder -> Explorer Menu -> TortoiseSVN -> Branch/Tag
    • select Tag URL: like http://svn.instance.org/repository/component/ Tags /tagname (i.e v1.0)
    • add comments
  • Tag is created (really a pointer)
  • Can checkout Tags folder - but not recommended
  • Usually work on the head revision in trunk
    • until experimental or multiple people working on code - then branches


Subversion on Linux: Command Line Instructions

Help for SVN commands: 

% svn help import
% svn help mkdir

Import Code to Subversion 

% svn import -m "Your Import Message" /path/to/FileOrFolderName \
      http://svn.instance.org/repository/component/trunk/folder/filename
Authentication realm: <http://svn.nccoos.org:80> nccoos
Password for 'uid':
Adding         file.ext

Committed revision 47.

Make Branches/Tags Subfolders 

% svn mkdir http://svn.instance.org/repository/component/branches
% svn mkdir http://svn.instance.org/repository/component/tags

Create a Working Copy - SVN Checkout 

% svn co http://svn.instance.org/repository/component/trunk/Folder \
      /path/to/workingcopy

Commit Changes Back to the Repository 

% svn update /path/to/WorkingCopy
% svn ci -m "comments" /path/to/WorkingCopy/