raw string in Python

 Normal strings use the backslash character as an escape character for special characters (like newlines):

>>> print('this is \n a test')

this is 

 a test

The r prefix tells the interpreter not to do this:

>>> print(r'this is \n a test')

this is \n a test

>>> 

This is important in regular expressions, as you need the backslash to make it to the re module intact - in particular, \b matches empty string specifically at the start and end of a word. re expects the string \b, however normal string interpretation '\b' is converted to the ASCII backspace character, so you need to either explicitly escape the backslash ('\\b'), or tell python it is a raw string (r'\b').

Ref: https://stackoverflow.com/questions/21104476/what-does-the-r-in-pythons-re-compiler-pattern-flags-mean/21104539#:~:text=According%20to%20this%20the%20%22r,literal%20prefixed%20with%20'r'.

byte and str in Python

 To store anything in a computer, you must first encode it, i.e. convert it to bytes. For example:

• If you want to store music, you must first encode it using MP3, WAV, etc.
• If you want to store a picture, you must first encode it using PNG, JPEG, etc.
• If you want to store text, you must first encode it using ASCII, UTF-8, etc.

MP3, WAV, PNG, JPEG, ASCII and UTF-8 are examples of encodings. An encoding is a format to represent audio, images, text, etc in bytes.

In Python, a byte string is just that: a sequence of bytes. It isn't human-readable. Under the hood, everything must be converted to a byte string before it can be stored in a computer.

On the other hand, a character string, often just called a "string", is a sequence of characters. It is human-readable. A character string can't be directly stored in a computer, it has to be encoded first (converted into a byte string). There are multiple encodings through which a character string can be converted into a byte string, such as ASCII and UTF-8.

'I am a string'.encode('ASCII')

The above Python code will encode the string 'I am a string' using the encoding ASCII. The result of the above code will be a byte string. If you print it, Python will represent it as b'I am a string'. Remember, however, that byte strings aren't human-readable, it's just that Python decodes them from ASCII when you print them. In Python, a byte string is represented by a b, followed by the byte string's ASCII representation.

A byte string can be decoded back into a character string, if you know the encoding that was used to encode it.

b'I am a string'.decode('ASCII')

The above code will return the original string 'I am a string'.

Encoding and decoding are inverse operations. Everything must be encoded before it can be written to disk, and it must be decoded before it can be read by a human.

Ref: https://stackoverflow.com/questions/6224052/what-is-the-difference-between-a-string-and-a-byte-string#:~:text=In%20Python%2C%20a%20byte%20string,is%20a%20sequence%20of%20characters.

Variadic Function in Python

 Variadic functions can accept  a variable number of arguments. 

 Naming convention: *args for positional input parameters and **kwargs for keyword input parameters.

 

# Function definition
def foo(*args, **kwargs):
    return args, kwargs

# Function calls
foo(1, 2, eleven=11, twelve=12)
# Output
# ((1, 2), {'eleven': 11, 'twelve': 12})

foo(*range(5,7), **{'thirteen': 13})
# Output
# ((5, 6), {'thirteen': 13})

mylist = [3,4]
mydict = {'fourteen': 14}
foo(*mylist, **mydict)
# Output
# ((3, 4), {'fourteen': 14})

Factory Design Pattern in Java

Code Example of Factory Design Pattern in Java:

Let’s see an example of how factory pattern is implemented in Code.We have requirement to create multiple currency e.g. INR, SGD, USD and code should be extensible to accommodate new Currency as well. Here we have made Currency as interface and all currency would be concrete implementation of Currency interface. Factory Class will create Currency based upon country and return concrete implementation which will be stored in interface type. This makes code dynamic and extensible.

Here is complete code example of Factory pattern in Java:

interface Currency {

       String getSymbol();

}

// Concrete Rupee Class code

class Rupee implements Currency {

       @Override

       public String getSymbol() {

              return "Rs";

       }

}

// Concrete SGD class Code

class SGDDollar implements Currency {

       @Override

       public String getSymbol() {

              return "SGD";

       }

}

// Concrete US Dollar code

class USDollar implements Currency {

       @Override

       public String getSymbol() {

              return "USD";

       }

}

// Factroy Class code

class CurrencyFactory {

       public static Currency createCurrency (String country) {

       if (country. equalsIgnoreCase ("India")){

              return new Rupee();

       }else if(country. equalsIgnoreCase ("Singapore")){

              return new SGDDollar();

       }else if(country. equalsIgnoreCase ("US")){

              return new USDollar();

        }

       throw new IllegalArgumentException("No such currency");

       }

}

// Factory client code

public class Factory {

       public static void main(String args[]) {

              String country = args[0];

              Currency rupee = CurrencyFactory.createCurrency(country);

              System.out.println(rupee.getSymbol());

       }

}

Read more: http://javarevisited.blogspot.com/2011/12/factory-design-pattern-java-example.html#ixzz2MoRNj6pT

Binary Tree

/**
 * Write a description of class BinaryTree here.
 *
 * @author (your name)
 * @version (a version number or a date)
 */
public class BinaryTree
{
   // Root node pointer. Will be null for an empty tree.
  private Node root;

  /**
   Creates an empty binary tree -- a null root pointer.
  */
  public void BinaryTree() {
    root = null;
  }

  /**
   Returns true if the given target is in the binary tree.
   Uses a recursive helper.
  */
  public boolean lookup(int data) {
    return(lookup(root, data));
  }

  /**
   Recursive lookup  -- given a node, recur
   down searching for the given data.
  */
  private boolean lookup(Node node, int data) {
    if (node==null) {
      return(false);
    }

    if (data==node.data) {
      return(true);
    }
    else if (data      return(lookup(node.left, data));
    }
    else {
      return(lookup(node.right, data));
    }
  }

  /**
   Inserts the given data into the binary tree.
   Uses a recursive helper.
  */
  public void insert(int data) {
    root = insert(root, data);
  }

   /**
   Recursive insert -- given a node pointer, recur down and
   insert the given data into the tree. Returns the new
   node pointer (the standard way to communicate
   a changed pointer back to the caller).
  */
  private Node insert(Node node, int data) {
    if (node==null) {
      node = new Node(data);
    }
    else {
      if (data <= node.data) {
        node.left = insert(node.left, data);
      }
      else {
        node.right = insert(node.right, data);
      }
    }

    return(node); // in any case, return the new pointer to the caller
  }

  private class Node
  {
   int data;
   Node left;
   Node right;


   Node(int newData) {
      left = null;
      right = null;
      data = newData;
   }
  }
}

Source: http://cslibrary.stanford.edu/110/BinaryTrees.html

Resetting Lost ESXi password

Resetting Lost ESXi password

Recovering from lockdown mode, a corrupt vCenter, and no DCUI

Recovering from lockdown mode, a corrupt vCenter, and no DCUI

How to reset the root password of VMware ESXi 4.1 and 5.0 | Bernhard Bock

How to reset the root password of VMware ESXi 4.1 and 5.0 | Bernhard Bock

PERL Module: I don't have permission to install a module on the system!

If you don't have root permission you will not be able to install a module in the usual place on a shared user system. If you do not have root access you may get errors like:

$ make install
Warning: You do not have permissions to install into
/usr/local/lib/perl5/site_perl/5.005/i386-freebsd at
/usr/libdata/perl/5.00503/ExtUtils/Install.pm line 62.
mkdir /usr/local/lib/perl5/site_perl/5.005/CGI/Simple:
Permission denied at /usr/libdata/perl/5.00503/ExtUtils/Install.pm line 120
*** Error code 2
This is easy to get around. You just install it locally in your home directory. Make a directory called say /lib in your home directory like this:

# first navigate to your home directory
$ cd ~
# now make a directory called lib
# on UNIX
$ mkdir lib
# on Win32
C:\> md lib
Now you have a directory called ~/lib where the ~ represents the path to your home dir. ~ literally means your home dir but you knew that already. All you need to do is add a modifier to your perl Makefile.PL command

$ perl Makefile.PL PREFIX=~/lib LIB=~/lib
This tell MakeMaker to install the files in the lib directory in your home directory. You then just make/nmake as before. To use the module you just need to add ~/lib to @INC. See Simple Module Tutorial for full details of how. In a nutshell the top of your scripts will look like this:

#!/usr/bin/perl -w
use strict;
# add your ~/lib dir to @INC
use lib '/usr/home/your_home_dir/lib/';
# proceed as usual
use Some::Module;

Source: http://www.perlmonks.org/?node_id=128077

Factory methods in Java

This story appeared on JavaWorld at
http://www.javaworld.com/javaworld/javaqa/2001-05/02-qa-0511-factory.html


Factory methods

How do you employ factory methods to your best advantage?

By Tony Sintes, JavaWorld.com, 05/11/01

While going through "Polymorphism in its purest form," I saw the unfamiliar term factory method. Could you please describe what a factory method is and explain how I can use it?

Factory method is just a fancy name for a method that instantiates objects. Like a factory, the job of the factory method is to create -- or manufacture -- objects.

Let's consider an example.

Every program needs a way to report errors. Consider the following interface:

Listing 1

public interface Trace {
// turn on and off debugging
public void setDebug( boolean debug );
// write out a debug message
public void debug( String message );
// write out an error message
public void error( String message );
}


Suppose that you've written two implementations. One implementation (Listing 2) writes the messages out to the command line, while another (Listing 3) writes them to a file.

Listing 2

public class FileTrace implements Trace {

private java.io.PrintWriter pw;
private boolean debug;
public FileTrace() throws java.io.IOException {
// a real FileTrace would need to obtain the filename somewhere
// for the example I'll hardcode it
pw = new java.io.PrintWriter( new java.io.FileWriter( "c:\trace.log" ) );
}
public void setDebug( boolean debug ) {
this.debug = debug;
}
public void debug( String message ) {
if( debug ) { // only print if debug is true
pw.println( "DEBUG: " + message );
pw.flush();
}
}
public void error( String message ) {
// always print out errors
pw.println( "ERROR: " + message );
pw.flush();
}
}


Listing 3

public class SystemTrace implements Trace {
private boolean debug;
public void setDebug( boolean debug ) {
this.debug = debug;
}
public void debug( String message ) {
if( debug ) { // only print if debug is true
System.out.println( "DEBUG: " + message );
}
}
public void error( String message ) {
// always print out errors
System.out.println( "ERROR: " + message );
}
}


To use either of these classes, you would need to do the following:

Listing 4

//... some code ...
SystemTrace log = new SystemTrace();
//... code ...
log.debug( "entering loog" );
// ... etc ...


Now if you want to change the Trace implementation that your program uses, you'll need to edit each class that instantiates a Trace implementation. Depending upon the number of classes that use Trace, it might take a lot of work for you to make the change. Plus, you want to avoid altering your classes as much as possible.

A factory method lets us be a lot smarter about how our classes obtain Trace implementation instances:

Listing 5

public class TraceFactory {
public static Trace getTrace() {
return new SystemTrace();
}
}


getTrace() is a factory method. Now, whenever you want to obtain a reference to a Trace, you can simply call TraceFactory.getTrace():

Listing 6

//... some code ...
Trace log = new TraceFactory.getTrace();
//... code ...
log.debug( "entering loog" );
// ... etc ...


Using a factory method to obtain an instance can save you a lot of work later. In the code above, TraceFactory returns SystemTrace instances. Imagine again that your requirements change and that you need to write your messages out to a file. However, if you use a factory method to obtain your instance, you need to make only one change in one class in order to meet the new requirements. You do not need to make changes in every class that uses Trace. Instead you can simply redefine getTrace():

Listing 7

public class TraceFactory {
public static Trace getTrace() {
try {
return new FileTrace();
} catch ( java.io.IOException ex ) {
Trace t = new SystemTrace();
t.error( "could not instantiate FileTrace: " + ex.getMessage() );
return t;
}
}
}


Further, factory methods prove useful when you're not sure what concrete implementation of a class to instantiate. Instead, you can leave those details to the factory method.

In the above examples your program didn't know whether to create FileTrace or SystemTrace instances. Instead, you can program your objects to simply use Trace and leave the instantiation of the concrete implementation to a factory method.

About the author

Tony Sintes is a principal consultant at BroadVision. A Sun-certified Java 1.1 programmer and Java 2 developer, he has worked with Java since 1997.

Implementing Callback in Java

interface InterestingEvent {
 // This is just a regular method so it can return something or
    // take arguments if you like.
    public void interestingEvent ();

}


class CallMe implements InterestingEvent {
    public CallMe ()
    {
     // en = new EventNotifier (this);
    } 
    // Define the actual handler for the event.
    public void interestingEvent ()
    {
     System.out.println("Wow!  Something really interesting must have occurred!");
    // Do something...
    } 
    //...

}


class EventNotifier {
    private InterestingEvent ie;
    private boolean somethingHappened; 
    public EventNotifier (InterestingEvent event)
    {
    // Save the event object for later use.
    ie = event; 
    // Nothing to report yet.
    somethingHappened = false;
    } 
    //...  
    public void doWork ()
    {
    somethingHappened = true; 
    // Check the predicate, which is set elsewhere.
    if (somethingHappened)
        {
        // Signal the even by invoking the interface's method.
        ie.interestingEvent ();
        }
    //...
    } 
    // ...


}

public class TestCallback {
    
    public static void main(String[] args) {
     CallMe me = new CallMe();
  EventNotifier event = new EventNotifier(me);
  event.doWork();
    }
}

Source: href="http://www.javaworld.com/javaworld/javatips/jw-javatip10.html

Array of Pointers

A pointer can only point to a single object such as char and int; whereas an array of pointers can point to several objects. For example,

    // void printName(char *p[])

void printName(char **p)  

{

printf("Name1: %s\n", p[0]);
printf("Name2: %s\n", p[1]);
}




int main(int argc, char *argv[])
{
char name1[] = "Name 1";
char name2[] = "Name 2";
char *p[2];


p[0] = name1;
p[1] = name2;


printName(p);


}

Simple File Input

Simple File Input

C++ programs can read and write files in many ways. For the sake of simplicity and uniformity, files are read and written using the same stream method already introduced for keyboard input. For this page, we first need a test data file with known contents. Please compile and run the following program:

#include 
#include 

using namespace std;

int main()
{
 ofstream ofs("data.txt");
 for(int i =1;i <= 10;i++) {
  ofs << "This is line " << i << endl;
 }
 return 0;
}
    

If you want to see what this program's output is, simply examine the contents of the file "data.txt" that this program creates when run. Not visible in the output file are some "control characters." A "control character" is a character that, instead of printing a symbol, causes an action to take place, like moving down to the next line on the display. A regular character is simply printed. A control character causes an action. Here are some common control characters, their symbols, and what they do:

Linefeed	'\n'	Causes the printing position to move to a new line
Tab	'\t'	Causes the printing position to advance to a fixed column
Bell	'\a'	Causes a bell to ring (most platforms)

These special symbols can be used alone or in quoted strings to format the display:

 cout << "This is a test line\n\n";
    

This example will print a line followed by two linefeeds, which assures one blank line appears before the next line is printed. Q: If I can add "\n" to the text of my printed lines, why use the special operator "endl" as in the example above?

The operator "endl" does two things. It (1) causes a newline to be printed, and it (2) causes the output to appear immediately.

In C++, input and output streams are "buffered." This means characters are read and written in groups, for the sake of efficiency. When keyboard input is being accepted, an entire line is read at once, which is why the user must press "Enter" to move along. When program data is being written, it is normally emitted in chunks. To force output to be emitted at a particular time, either use "endl" as in the above example, or do this:

 cout << "This will appear right away." << flush;
    

The operator "flush" causes immediate output, which means these two lines are equivalent:

 cout << endl;
 cout << "\n" << flush;
    

Now let's read our data file in an obvious way — line by line. The following program contains the single most common student error in file reading. See if you can spot what the error is, what mistake it creates, and why. To reduce any chance for confusion, the error is in red :

#include 
#include 
#include 

using namespace std;

// this program contains an error

int main()
{
 ifstream ifs("data.txt");
 string line;
 // error in stream test
 while(!ifs.eof()) {
  getline(ifs,line);  cout << "[ " << line << " ]" << endl;
 }
 return 0;
}
    

When you run this program, you will see a blank line is printed after the last valid data line. This is caused by the program error. The error is attempting to test the stream for "end-of-file" without also trying to read it:

 while(!ifs.eof()) {
    

Remember: a C++ stream can have any origin — a file, a network connection, a keyboard, or any other source. Therefore the stream cannot detect that the data has ended until a read attempt fails. Because of this, a program that tests for end-of-file without reading, then reads without testing, always fails. A successful program always tests and reads at once. Here is a corrected version of the program:

#include 
#include 
#include 

using namespace std;

int main()
{
 ifstream ifs("data.txt");
 string line;
 while(getline(ifs,line)) {
  cout << "[ " << line << " ]" << endl;
 }
 return 0;
}
    

In the next example, we will use the stream extraction operator ">>" to read our file. Compile and run this program (it also has an error ):

#include 
#include 
#include 

using namespace std;

int main()
{
 ifstream ifs("data.txt");
 string word1, word2, word3;
 int num;
 while(ifs >> word1 >> word2 >> word3 >> num) {
  cout << "[ "
  << word1
  << word2
  << word3
  << num
  << " ]"
  << endl;
 }
 return 0;
}

    

Why does the error cause the output to look all squeezed together? To answer this question, we need to look at how the stream extraction operator ">>" works. The extraction operator is actually a very sophisticated tool. It knows what kind of variable is receiving the data, and it conducts itself accordingly. It works like this:

• Phase 1 (search):
  • Read characters.
  • If a character is "whitespace" (control characters or spaces), discard it.
  • If a non-whitespace character appears that is not appropriate to the target variable, stop, indicate an error, "break" the stream.
  • If a character is appropriate to the target variable --
    • For integer variables, any of "+-0123456789".
    • For float/double variables, any of "+-.0123456789e" in a prescribed order.
    • For string variables, any non-whitespace characters.
    — begin phase 2.
• Phase 2 (read):
  • Read and accept characters that are appropriate to the target variable.
  • If a character appears that is not appropriate to the target variable, either whitespace or some other character, don't discard it, stop reading, no error.

If you can commit this sequence of events to memory, it will greatly aid your stream programming. Using the extraction operator is called "formatted reading." It is called "formatted" because the input is expected to have a particular format — groups of characters meant to be received by particular variable types, separated by whitespace. This kind of reading is ideal for text files that contain different kinds of data, data that is separated by whitespace. Here is another common student error — mixing the extraction operator and getline() in the same program, without appropriate safeguards. Compile and run this program:

#include 
#include 
#include 

using namespace std;

int main()
{
 ifstream ifs("data.txt");
 string word1, word2, word3, line;
 int num;
 // read a line using the extraction operator
 if(ifs >> word1 >> word2 >> word3 >> num) {
  cout << "[ "
  << word1 << ' '
  << word2 << ' '
  << word3 << ' '
  << num << " ]"
  << endl;
 }
 // read a line using getline
 if(getline(ifs,line)) {
  cout << "[" << line << "]" << endl;
 } return 0;
}
    

Why does this program fail — why can't it read the file's second line? Think:

1. In Phase 2, the extraction operator reads characters until it encounters whitespace, then it stops without discarding any of the whitespace.
2. getline() only reads until it encounters a linefeed.

Unfortunately for our test program, the whitespace character that is left behind by the extraction operator is a linefeed. getline() reads this single linefeed and stops without reading the line that follows it. The solution is to remove the linefeed that was left behind by the extraction operator. Here is the corrected program:

#include 
#include 
#include 

using namespace std;

int main()
{
 ifstream ifs("data.txt");
 string word1, word2, word3, line;
 int num;
 // read a line using the extraction operator
 if(ifs >> word1 >> word2 >> word3 >> num) {
  cout << "[ "
  << word1 << ' '
  << word2 << ' '
  << word3 << ' '
  << num << " ]"
  << endl;
 } // discard whitespace
 ifs.ignore(10000,'\n');
 // read a line using getline
 if(getline(ifs,line)) {
  cout << "[ " << line << " ]" << endl;
 } return 0;
}

Source: http://www.arachnoid.com/cpptutor/student2.html
    

Perl: BEGIN and END

Perl Script	Output
BEGIN { print "Birth of a process...\n"; } print "Life of a process...\n"; die "Murder of a process...\n"; END { print "Death of a process...\n"; }	Birth of a process... Life of a process... Murder of a process... Death of a process... bash-2.03$ bash-2.03$
Perl Script	Output
BEGIN { print "Birth still runs !\n"; } This won't compile; END { print "Death still runs !\n"; }	Birth still runs ! Can't locate object method "This" via package "won::t" (perhaps you forgot to load "won::t"?) at 1.pl line 5. Death still runs !
Perl Script	Output
die "Bye"; END { print "End 1\n" } BEGIN { print "Begin 1\n"; } END { print "End 2\n" } BEGIN { print "Begin 2\n"; }	Begin 1 Begin 2 Bye at 1.pl line 1. End 2 End 1

BEGIN

Run some code as soon as it has all been read in by the parser and before the rest compiles.

END

Run some code as late as possible. When the interpreter is ready to shut down.

Source: http://www.compuspec.net/reference/language/perl/BEGIN_and_END.shtml

How to implement callback functions in Java?

The callback function is excutable code that is called through a function pointer. You pass a callback function pointer as an argument to other code, or register callback function pointer in somewhere. When something happens, the callback function is invoked.

C/C++ allow function pointers as arguments to other functions but there are no pointers in Java. In Java, only objects and primitive data types can be passed to methods of a class. Java's support of interfaces provides a mechanism by which we can get the equivalent of callbacks. You should declare an interface which declares the function you want to pass.

The collections.sort(List list, Comparator c) is an example of implementing callback function in Java. The c is an instance of a class which implements compare(e1, e2)method in the Comparator interface. It sorts the specified list according to the order induced by the specified comparator. All elements in the list must be mutually comparable using the specified comparator.

Use inner classes to define an anonymous callback class, instantiate an anonymous callback delegate object, and pass it as a parameter all in one line. One of the common usage for inner classes is to implement interfaces in event handling.

class SomePanel extends JPanel {

    private JButton    myGreetingButton = new JButton("Hello");
    private JTextField myGreetingField  = new JTextField(20);

    private ActionListener doGreeting = new ActionListener {
            public void actionPerformed(ActionEvent e) {
               myGreetingField.setText("Hello");
            }
 };

    public SomePanel() {
        myGreetingButton.addActionListener(doGreeting);
        // . . . Layout the panel.
    }

}

Source: http://www.xyzws.com/javafaq/how-to-implement-callback-functions-in-java/157

UCM with SVN

This article describes how to apply UCM with the SVNVCS.

Contents

• Introduction
  • About VCS
  • About ITS
  • About UCM
  • About SVN
• How to perform UCM using SVN
  • Projects
  • Streams
  • Components
  • Issue Tracking

Introduction

Proper change tracking and version control is vital for the success of any software project. This article describes the UCM approach to change control and how to implement it using SVN.

About VCS

Version Control Systems are vital for the success of any software project. Programmers must have possibilities to experiment with new ideas without impacting other team members and general development. Changes should be thoroughly tested before they are delivered to a common code base. It must be possible to see when and by whome each change to the software was introduced.

More elaborate environments require even more. Different variants of a product must be developed independently of each other. For instance, while one subteam is preparing the current code for a release and polishes away the last known bugs, another team at the same time continues development for a new version. Changes by the development team shouldn't affect the release team.

A relatively new approach requires that different variants of a product are made up of different components, which are tracked individually by the version control system. This is component based version control.

About ITS

Issue Tracking Systems, also known as Bugtrackers or Change Request Managers (there are slight differences between these terms which are out of scope of this document), allow developers and other participants in projects, like project managers, trackers, testers and quality assurance, to track the state about issues like open bugs or requested features.

About UCM

Unified Change Management is a relatively young approach to version control. It's main attributes are:

• Stream-oriented development and integration
• Activities
• Component oriented version control
• Close coupling with an issue tracking system

UCM gained much popularity in the context of IBM Rational ClearCase UCM, which is an extension of ClearCase to support the UCM approach of version control.

About SVN

Subversion is a light-weight VCS meant to replace CVS. CVS was the most popular VCS in open source development for a very long period of time. CVS is very easy to handle and for a version control system easy to understand. Besides that CVS also is extremely fast, in LAN environments as well as in WAN environments. Even over the Internet, CVS is very fast.

As a replacement for CVS, SVN is meant to provide all positive features of CVS as well as several new features. Apart from being more modern in supporting versioning of directories and file attributes, SVN provides three main attributes / advantages over CVS which also significantly differ SVN from most other VCS:

Versions are applied to the repository, not single files or directories.

This makes it extremely easy to identify all matching files and directories for a specific version of a particular file.

SVN has cheap copies.

The cheap copy mechanism replaces previous mechanisms for branching, tagging or labelling. A cheap copy is a copy of a whole directory tree made inside the repository. Branches, tags and labels are implemented using these cheap copies. The cheap copies are named cheap because creating a cheap copy is an operation with very very little overhead. It's nothing else like "I'm a copy of path/to/foo revision n".

This also means that in SVN, attributes like branches, labels or tags merely are directories. This leaves much individual flexibility on how to organize version management to the projects, they can adapt it on their needs.

svn:externals

svn:externals is a property which allows to link to other repositories or other locations in the same repository.

How UCM works in general

First of all, development is no longer parted in a main branch plus lots of changes to it, but changes are grouped in multiple ways. All changes that together make up a single feature or bug fix or similar group are grouped as "activity". An activity appears as issue item in the issue tracker, that means an activity is associated with an issue tracker item and vice versa. Activities are performed on a development stream, which is a copy of the integration stream, and then "delivered" (merged) into that integration stream.

Original UCM concepts

This subchapter describes the original UCM concepts as they were applied in ClearCase UCM. The goal is to be able to apply UCM with any version control system, particularly Subversion. For that, it is necessary to understand the intentions of UCM and its concepts, and why it is implemented in ClearCase the way it is.

ClearCase limitations

ClearCase is a very old-fashioned version control system. Basically ClearCase is RCS with improved multi-user and directory support, a network file system (as if there weren't already enough of them) and database backend. The elementary concepts staid the same over time. That means:

• Revisions are done on a per-element basis.
• There is no low-level support for atomic changesets.
• Conflicts are prevent with locking (reserved checkout).

ClearCase UCM is built on top of ClearCase. ClearCase UCM has to take into account these limitations and live with them. ClearCase UCM has to hide these limitations.

ClearCase UCM glossary

Activity

An activity is a group of changes that belong together. Purpose:

• They link changes together, putting single checkout / checkin operations into the context of a group called Activity.
• They link changes with an issue tracker.

Baseline

A baseline is a consistent version state of a component. Purpose:

• Keep those revisions of files and directories together which belong together.

Component

A component is a group of files and directories which are managable as single unit to be baselined. Purpose:

• Keep those files and directories together which belong together.

Composite baseline

A composite baseline is the combination of multiple baselines from several components. Usually, a project has a composite baseline which sums up the baselines for all participating components. Purpose:

• Keep those revisions of components in a project together which belong together.

Delivery

Delivery is the operation of merging changes from a child Stream to a parent Stream (usually the Integration Stream). Purpose:

• Perform merges on groups of files and directories instead of single files and directories.
• Keep changesets together.

Integration Stream

An Integration Stream is a Stream with child streams. For a Project, at least one Stream exists, which is its Integration Stream. Purpose:

• Have a branch on which all changes that are mature enough are available.

Project

A Project is a configuration of an Integration Stream and participating components in specific baselines. A project is configured by a composite baseline. Purpose:

• Prevent conflicts between different development goals of different projects.
• Allow the selection of components from a pool.

Rebase

Rebase is the operation of copying changes from a parent Stream (usually the Integration Stream) to a child Stream. Purpose:

• Perform merges on groups of files and directories instead of single files and directories.
• Keep changesets together.

Stream

A Stream is an isolated branch of development. Unless it is an integration Stream, a Stream always has a parent. Activities will only affect the stream for which they were performed. Streams are actively synchronized with each other by explicit Delivery and Rebase operations. Purpose:

• Allow changes under version control but in a controlled manner.
• Keep changesets together.

Putting UCM on a more abstract level

Many of the concepts which ClearCase UCM puts on top of ClearCase are only explicitely necessary to cope with the limitations of ClearCase. To be able to apply UCM with any version control system, it is necessary to understand the intentions of UCM and its concepts, and why it is implemented in ClearCase the way it is.

Goals on an abstract level:

• Developers shall be able to perform changes without interfering with other developers.
• Changes shall be controlled and tracked.
• Projects shall be setup in a way that they are built on reusable components.
• The reusable components shall be setup in a way that allows changing them without interfering with other projects.

Simple mapping UCM between ClearCase UCM and Subversion

For most of what is done with ClearCase UCM, the following simple mapping may already be sufficient.

Goal	ClearCase UCM	Subversion
Atomic changesets (small changes)	Activity	commit
Atomic changesets (large changes)	Activity	directory copy, merge
Consistent revisions	Baseline	implicit (path + repository revision)
Reusable file group	Component	directory copy
Consistent project version	Composite baseline	implicit (path + repository revision)
Integrate developer work (small change)	Delivery	commit
Integrate developer work (large change)	Delivery	merge --reintegrate
Mature main branch	Integration Stream	implicit (e.g. policy for trunk)
Consistent overall setup	Project	directory
Up-to-date developer copy (small change)	Rebase	update
Up-to-date developer copy (large change)	Rebase	merge
Create place for independent developer work (small change)	create Stream	checkout
Create place for independent developer work (large change)	create Stream	directory copy

Just like having no concept of tags and labels, or branches, Subversion has no concept projects, components, streams and activities. Actually, that's good news. The concept of Subversion is to work with cheap copies. Originally, cheap copies were designed to be the Subversion approach to tagging and branching. Interestingly, the more abstract approach of cheap copies is not only a simple yet superior way to resemble tags and labels, and branches. It also is a simple way to resemble projects, components, streams and activities.

At this point, let me also have a word on whether you should have one large repository or multiple small repositories. Subversion is significantly faster than ClearCase. And because commits are path-based, it can use path-based transaction isolation. That means multiple proceses can safely work on the same repository at the same time. They will only delay each other if they affect the same path.

Also, projects like KDE show that it's possible to use Subversion for very large development projects with large development teams. At the time of this writing, the KDE subversion repository was at revision 969346.

How to perform UCM using ClearCase

As already mentioned above, there's a special version of ClearCase called ClearCase UCM. When applying UCM with ClearCase, you'll notice the following things:

• Working with ClearCase when the server is offline is near impossible.
• Dynamic views don't work with the server being offline.
• Snapshot views work with the server being offline, but this is very limited. Even restoring a file to its original version requires server access.
• VOB symbolic links don't work properly with snapshot views.
• Over WANs / Internet and VPNs, ClearCase is awfully slow.
• When you're working on your own stream anyway, you'll begin to ask yourself why this obsolete checkin / checkout locking mechanism is still required.
• ClearCase will create new file revisions of all files you've touched, even if you don't actively create new versions. A file, once touched, will grow new versions for every rebase you perform.
• Activities are somehow lost once they're integrated. They are visible on the development stream but they don't become visible on the integration stream. Instead, Deliveries and Rebases appear as separate activities. The faster the development process, the higher the percentage of these artificial pseudo-activities compared to real development activities.

How to perform UCM using SVN

Most of the UCM concepts can be provided by directories and cheap copies in SVN: projects, components, streams and, if you want to, activities. That means the implementation of UCM on top of Subversion is very flexible. It requires only little discipline, so no additional scripts should be required. Also, subversion is much more likely to forgive errors or mistakes in the setup. And it's easy to restructure the repository afterwards, so you can even turn a non-UCM-repository into a UCM-repository.

The flexibility of Subversion also means that the following is just one out of many possible ways to implement UCM on top of Subversion.

Original Subversion approach: TTB

• /
  • trunk/
  • tags/
  • branches/

This is the original subversion approach, also known as TTB structure. It already has a lot in common with UCM. Small development activities are separated by commits, large development activities can be performed on separate branches.

Single project / component approach

• /
  • trunk/
  • tags/
  • branches/
  • streams/ (← This is new compared to the classical approach in subversion)

This is a slightly modified approach, which I call TTBS (you guessed ;-). It separates streams / developer branches from (release) branches.

Distinct Multi-Project approach

• /
  • projectname
    • trunk/
    • tags/
    • branches/
    • streams/ (← This is new compared to the classical approach in subversion)

Often, multiple projects shall be supported in the same repository. For that, it is a good idea (and common practice) to have the projects on top level and the TTB(S) structure below that.

If your component model works on mature objects only, which is the case for many Java projects, this is enough: If one project depends on the other, it waits for a release (.jar, .war, .ear or so).

However, in some development environments, it takes days or even weeks for a project to make a release for just one minor change. In such an environment, if project A depends on project B, project A might be better off with a copy of project B. That's the next approach.

Establish Reuse

Reuse means to have something available which was created elsewhere. For reuse, subversion offers two possibilities, both of which work fine on directories (and single files, if wanted):

• cheap copies
• svn:externals

If you want to reuse something in an unchanged form, the best way of doing that is using svn:externals. If you want to change it, the best way of doing that is using cheap copies (svn cp).

Separate component / project approach

• /
  • projects/
    • projectname
      • trunk/
      • tags/
      • branches/
      • streams/
  • components/
    • componentname
      • trunk/
      • tags/
      • branches/
      • streams/

A component can be made visible anywhere within a project, either as cheap copy or as external.

How to apply UCM to SVN - The Commands

Creating a Project or Component

As a project or component is just a directory, it's created like trunk: Create the directory, if you start from scratch, or import something that you already have.

Creating a Stream

To create a Stream, create a copy from what you want to create a stream for, and place that copy in streams/.

Synopsis: svn cp parentPath streams/streamName

Example stream from trunk: svn cp -m "Creating Stream barBuzz." http://svn.mynet.local/projectFoo/trunk http://svn.mynet.local/projectFoo/streams/barBuzz

Rebasing a Stream

Starting with Subversion 1.5, rebasing a stream becomes very simple. Use the svn merge command for that.

Synopsis: svn merge parentPath

Example: cd barBuzz; svn merge http://svn.mynet.local/projectFoo/trunk

Delivering a Stream

First you rebase your stream to make sure you deliver a consistent set. Then you go to trunk and deliver using the svn merge --reintegrate command.

Synopsis: cd parentPath ; svn merge --reintegrate streams/streamName

Example: cd ../../trunk ; svn merge --reintegrate http://svn.mynet.local/projectFoo/streams/barBuzz

Once the deliver merge is done, check the delivery and if everything is okay, complete it by performing a commit.

As soon as the delivery is done, the stream should be removed. It is neither possible nor desired from a subversion point of view to reuse a stream once a delivery is done. If you want to continue working on that stream after the delivery, like some developers do in ClearCase UCM, just recreate the stream after deleting it.

Removing a Stream

To remove a stream, simply delete the stream like a normal directory. Removing a stream will not reduce the server load, it will just make the stream invisible, nothing more.

Example

This chapter describes an example of how development could work, and it also shows how a subversion repository can be transformed to the different parts of UCM as needed.

The beginning is an application called "JEduca", an application for creating and running tests based on multiple choice questions. The developers are Alice, Bob and Charly.

Alice creates a repository for them:

svnadmin create /home/alice/svnroot

The developers that begin to work on JEduca, execute:

mkdir ~/svn
cd ~/svn
svn checkout file:///home/alice/svnroot JEduca

Whenever they want to access changes that the others made, they perform a rebase:

svn up

Whenever they have finished an activity, they perform a delivery:

svn commit -m "Activity description."

After a few days, Alice and Bob want to create the first release, while Charly wants to continue working on new features. They decide it's time for a branch. Prior to branching they have to perform a small change to the repository and make it ready for branches: Bob executes:

files=$(echo *)
svn mkdir trunk tags branches
svn mv $files trunk/
svn commit -m "Created TTB structure."

The sandboxes now have should be relocated to trunk, because the position of trunk has virtually changed.

Now that the repository is raedy for branches, Bob creates the branch for the 0.1 release:

svn cp -m "Create release branch for 0.1." file:///home/alice/svnroot/trunk file:///home/alice/svnroot/branches/0.1

The story continues with the next update.

Source: http://www.riedquat.de/articles/UCMwithSVN