Version Control/Migration from MKS to SVN

The goal of this article is to describe how to migrate from MKS 7.4 to SVN.

As MKS 7.4 is no longer really supported, the need for migration is there, and SVN is of course a very natural move, and as this is not so common, I decided that it would be worth sharing with others, and also get feedback remarks from people having the same idea or having done the migration.

This migration rely on certain convention that we use for MKS. this might not be true for you, but anyway gives some idea how to customize it for you.

Main differences between MKS and SVN

In order to plan the migration, one has to understand the basic differences between MKS ans SVN.

• In MKS, a file belongs to a folder (or sub folder) and will never move from there: in SVN one can copy and revision from a project folder to another project folder, resulting in a "cheap copy" (a pointer).
• In MKS, a project is a special file having the "bill of material" (= member + revision) needed for this project. One can checkpoint a project (=snapshot it) to see how is the project at a given point in time. In SVN, a project is a structure of folder, traditionnaly starting with ./trunk; ./branches ./tags and instead of checkpointing a project, one copies all file in a tags/<rev> folder (again a cheap copy).

Basic concepts of the migration

The basic concept for the migration are:

• it must be possible to stop the migration at any time, but of course at any time an error ca occur. This gives the time to correct the error, and resume. As in our case, it will take several CPU month to migrate, this fact is very important.
• We will replay all operations that occurred from day one in MKS to SVN.
  • every check-in in MKS will be a new commit in SVN; even if more than one file where checked-in at the same time, we will make one commit for each file, to simplify the process. as in MKS, each file has its own revision number, we will associate the commit number with the member/revision, in order to retrieve it after. We will conserve the revision number in a property of the revision (and in the log)
  • every checkpoint will be and SVN COPY to create a cheap copy

Retrieve all files from the archives

In our implementation of MKS, all files archives are located in L:\ARCH\...

One must take care not to use files in mks.*.*.revs folders that are a way to store revision of a binary file.

The following fragment of Python code can do that (here it only prints the file names)

 f=open("L:\\test.txt","wt")
 for root,dirs,files in os.walk("l:\\arch"):
   if root.endswith(".revs"):
     dirs = []
     files = []
   for file in files:
     print(join(root,file),file=f)
 f=None

Extracting revisions from an archive

To retrieve the revision, I use the RLog.exe MKS command

 subprocess.Popen(["L:\\32bits\\mks74\\mksnt\\RLog.exe",win32api.GetShortPathName("L:\\ARCH\\L\\CUBE\\auto.pas")], stdout=subprocess.PIPE).communicate()[0]

Note that with MKS 7.3, there is an issue using short filename, so one HAS to use 7.4 command is order to extract all filenames (incuding the one that can only be expressed as shortfilename)

the result is (coded as CP1252 in our case). Here I use the """ notation and replaced all \r\n by a new line for better readability

   TestRLog="""
   Archive file: L:/ARCH/L/CUBE/auto.pas;\tWorking file: L:/REF/L/CUBE/auto.pas
   head:\t3.64
   branch:\t
   locks:  strict
   access list: 
   symbolic names: cube32_2: 3.63; cube32_1: 3.21; beta_cube_3: 3.21; 
                   cube_3: 3.14; 
   comment leader:\t" * "
   total revisions: 64; branches: 0; branch revisions: 0
   file format: text
   revision storage: RCS reverse deltas
   project: L:\\cube.pj;\r

cut...

   project: l:/ref/0736_5063_gen_consigne_922.pj;
   description:
   Introduction de la version 3.0\r
   \r
   
   ----------------------------
   revision 3.64\t
   date: 2010/01/13 14:27:07Z; author: pittet0t; state: dev; lines: +62 -61
   Adaptation Delphi 2010
   ----------------------------
   revision 3.63\t
   date: 2006/10/23 08:11:35Z; author: moret_d; state: dev; lines: +2 -3
   suppression d\'un appel a une proc\xe9dure vide de obflash
   ----------------------------
   revision 3.62\t
   date: 2006/04/27 11:27:35Z; author: moret_d; state: dev; lines: +8 -2
   ajoit directive pour cible 3g
   ----------------------------
   revision 3.61\t
   date: 2006/01/17 09:49:28Z; author: NICETA_S; state: dev; lines: +3 -1
   Corrig\xe9 assert si unit\xe9 obechean \xe9tait link\xe9e apr\xe8s auto. N\xe9cessite obechean.pas ver 3.91
   ----------------------------

cut...

   ----------------------------
   revision 3.1\t
   date: 1996/08/27 08:11:02Z; author: NICOLE; state: Exp; 
   Introduction de la version 3.0\r
   ----------------------------
   selected revisions: 64
   -------------------------------------------------------------------------------
   """

Issues with non ASCII characters in filenames

unfortunately MKS command lines do not handle non ascii characters properly (above 0x7F) so far, the only way that seems to work is to use short filenames (8.3)

convertion could be done with DIR (beeeaarkk)

Extraction of the history of a project

 Archive file: l:/ARCH/cubelibrary.pj;    Working file: C:/cubelibrary.pj
 head:    3.38
 branch:
 locks:
 access list:
 symbolic names: cubelibrary: 3.32;
 comment leader:
 total revisions: 236; branches: 15; branch revisions: 130
 file format: text
 revision storage: RCS reverse deltas
 project: l:/ref/bidon_13.pj;

..cut.. lists all project that include this project (more or less accurate: we don't use this information)

 description:

 ----------------------------
 revision 3.38
 date: 2010/12/06 15:24:13Z; author: goutem0f; state: dev; lines: +46 -40
 +l:/ref/cubelibrary.pj 2010.12.01 12:59:40 W. Europe Standard Time revision 3.37 by ros0J
 checkpointed, new revision 3.37

..cut.. list all added or droped file: we don't use this info, since we will ask for all details for each revision

 ----------------------------
 revision 3.37
 date: 2010/12/01 11:59:40Z; author: ros0J; state: dev; lines: +31 -25
 Status for F&K disribution 1.12.2010
 ----------------------------
 revision 3.36
 date: 2010/11/15 10:19:59Z; author: pittet0t; state: dev; lines: +125 -44
 +l:/ref/cubelibrary.pj 2010.10.29 10:40:28 W. Europe Daylight Time revision 3.35 by pittet0t
 checkpointed, new revision 3.35
 l:/ref/l/cubeimg/imgcoul/c0507068.gif drop
 l:/ref/l/cubeimg/imgcoul/c0507078.gif drop
 +l:/ref/cubelibrary.pj 2010.11.03 13:59:20 W. Europe Standard Time revision 3.35 by niceta0s
 ----------------------------
 revision 3.35
 date: 2010/10/29 08:40:27Z; author: pittet0t; state: dev; lines: +15 -2
 +l:/ref/cubelibrary.pj 2010.10.27 09:29:12 W. Europe Daylight Time revision 3.34 by pittet0t
 checkpointed, new revision 3.34
 ----------------------------
 revision 3.34
 date: 2010/10/27 07:29:11Z; author: pittet0t; state: dev; lines: +399 -227
 Ramené au niveau de la branche NewHMI
 ----------------------------
 revision 3.33.1.50
 date: 2010/11/04 08:29:31Z; author: pittet0t; state: dev; lines: +75 -46
 Avant vidage
 ----------------------------
 revision 3.33.1.49
 date: 2010/10/11 07:43:27Z; author: ros0J; state: dev; lines: +23 -22
 +l:/ref/cubelibrary.pj 2010.10.11 09:31:29 W. Europe Daylight Time revision 3.33.1.49 by ros0J from variant sandbox l:/cubevariants/cubelibrary_newhmi.pj
 checkpointed, new revision 3.33.1.49
 +l:/ref/cubelibrary.pj 2010.10.11 09:43:09 W. Europe Daylight Time revision 3.33.1.49 by ros0J from variant sandbox l:/cubevariants/cubelibrary_newhmi.pj
 l:/ref/l/tool32/dynapassdll.dll new-revision 1.3 was 1.2

...cut... We will find all revision down to the first

 ----------------------------
 revision 1.1
 date: 2003/05/23 14:38:18Z; author: ROS_JC; state: old;
 Initial Revision
 ----------------------------
 selected revisions: 236
 -------------------------------------------------------------------------------

Extraction of the list of member for one revision of a project

 C:\mks2svn>pj.exe -Pl:\ref\project.pj -k3.37  rlog -S%A[\t%r]  > project.pj_rlog