Source code management

From FreeCAD Documentation
Revision as of 20:40, 25 July 2019 by Vocx (talk | contribs) (The present document focuses on the use of Git for FreeCAD development. Compiling FreeCAD is described in Compiling.)

The main source code management tool for the FreeCAD project is Git, which can be easily installed in most operating systems from a package manager or directly from Git's website. You are advised to become familiar with Git before working with the FreeCAD source code directly. Visit the Git documentation page for the reference manual, as well as the Pro Git book to learn to use the system in a general way. The present document focuses on the use of Git for FreeCAD development. Compiling FreeCAD is described in Compiling.

While Git is primarily a terminal application, there are many graphical clients for it which facilitate working with branches, applying patches, and submitting pull requests to a master branch. Examples include git-cola, gitg, and GitKraken. Please see Developing FreeCAD with GitKraken for a cursory introduction to this tool.

Source code access

Everybody can access and get a copy of the FreeCAD source code, but only the FreeCAD project managers have write access to it. You can get a copy of the code, study it and modify it as you wish. If you want your changes to be included in the official source code, you need to perform a pull request against the master repository so that your modifications can be reviewed by one of the managers.

If your source code changes are significant, you are advised to explain them in the pull request section of the FreeCAD forum.

Official GitHub repository

The FreeCAD source code is hosted in Github, https://github.com/FreeCAD/FreeCAD

In order to contribute code, you need to have a GitHub account.

Setting your git username

Developers should commit code to their personal repository using their GitHub username. If that is not already set globally, you can set it locally for the current Git repository like this:

git config user.name "YOUR_NAME"
git config user.email GITHUB_USERNAME@users.noreply.github.com

Where "YOUR_NAME" represents your full name or nickname, used to identify the author of a particular commit, and GITHUB_USERNAME indicates your GitHub username.

You can now use some combination of git add and git commit to create one or more commits in your local repository.

Remote repositories

Please read What is the difference between origin and upstream on GitHub? (Stackoverflow) to help you understand the difference between origin and upstream in the context of Git. This section explains how to set the correct repositories for development. Essentially:

  • origin is your personal fork of the official FreeCAD repository, that is, https://github.com/GITHUB_USERNAME/FreeCAD
  • upstream is the official FreeCAD repository, that is, https://github.com/FreeCAD/FreeCAD

This distinction is important, as you should write code in your own copy of the repository first, before pushing those changes to the official repository.

Based on the above, there are two ways to setup your Git development environment:

  • 1st Method: fork on GitHub and clone your fork locally
  • 2nd Method: clone FreeCAD directly to your local machine, and adjust the remote servers

We recommend the 1st method because it's one step faster.

1st Method: Fork on GitHub and clone your fork locally

First you will fork the FreeCAD repository in GitHub, then clone this personal fork to your computer, and finally set the upstream repository.

  • Log in to your GitHub account.
  • Go to the official FreeCAD repository: https://github.com/FreeCAD/FreeCAD
  • In the top right of the page press the "Fork" button. This will create a personal copy of the FreeCAD repository under your GitHub username: https://github.com/GITHUB_USERNAME/FreeCAD
  • On your machine, clone your newly created FreeCAD fork. It will be created inside a directory freecad-source.
git clone https://github.com/GITHUB_USERNAME/FreeCAD.git freecad-source
  • Once the download is complete, enter the new source directory and set the upstream repository.
cd  freecad-source
git remote add upstream https://github.com/FreeCAD/FreeCAD.git
  • Confirm your remote repositories with git remote -v; the output should be similar to this
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (fetch)
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (push)
upstream	https://github.com/FreeCAD/FreeCAD.git (fetch)
upstream	https://github.com/FreeCAD/FreeCAD.git (push)
  • Now development can begin.

2nd Method: Clone FreeCAD directly to your local machine

First you will fork the FreeCAD repository in GitHub, however, you will clone the original FreeCAD repository to your local machine, and then alter your remotes via the terminal.

  • Log in to your GitHub account.
  • Go to the official FreeCAD repository: https://github.com/FreeCAD/FreeCAD
  • In the top right of the page press the "Fork" button. This will create a personal copy of the FreeCAD repository under your GitHub username: https://github.com/GITHUB_USERNAME/FreeCAD
  • Clone the original FreeCAD repository. It will be created inside a directory freecad-source.
git clone https://github.com/FreeCAD/FreeCAD.git freecad-source
  • Once the download is complete, enter the new source directory and set the origin repository.
cd freecad-source
git remote add origin https://github.com/GITHUB_USERNAME/FreeCAD.git
  • Then set up the upstream repository.
git remote add upstream https://github.com/FreeCAD/FreeCAD.git
  • Confirm your remote repositories with git remote -v; the output should be similar to this
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (fetch)
origin	https://github.com/GITHUB_USERNAME/FreeCAD.git (push)
upstream	https://github.com/FreeCAD/FreeCAD.git (fetch)
upstream	https://github.com/FreeCAD/FreeCAD.git (push)
  • Now development can begin.

If for some reason the remote repositories exist but point to the wrong address, you can remedy the situation by renaming the remote repository's name. For example, origin should point to your personal fork; if it is pointing to the original FreeCAD repository, change the name of this remote to upstream, and manually add the origin repository.

git remote rename origin upstream
git remote add origin https://github.com/GITHUB_USERNAME/FreeCAD.git
git remote -v

Git development process

Never develop on the master branch. Instead, create a branch for development, and then merge this branch to the master branch. Please read Git Branching and GitHub - Contributing to a project to learn more.

Branching

Instead of working on the master version of the code, best practices with Git recommend creating a new branch whenever you want to work on a new feature. Using a new branch is done in two steps, first your create the branch, and then you switch to it:

git branch myNewBranch
git checkout myNewBranch

Alternatively, use a single instruction:

git checkout -b myNewBranch

To see the branches in your project and the current branch, use the branch operation alone, or add -v or -vv for more information:

git branch
git branch -vv

After you've made changes and committed those changes use the log operation with the following options to visualize the branches

git log --oneline --decorate --graph --all

Committing

Once you are inside a new branch, edit the source files that you want with a text editor. To see which files were modified use the status and diff operations; when you are satisfied with the modifications, save the changes with the commit operation:

git status
git diff
git commit -a

Unlike SVN, you need to specifically tell which files to commit; use the -a option to save changes in all files that were altered. Your text editor, like nano or vim, will open to allow you to write a commit message.

Alternatively add the message in the commit itself

git commit -a -m "Fix the bug in the clone function."

Writing good commit messages

You should try to work in small steps, that is, commit often, after a small addition in your code. If you cannot summarize your changes in one sentence, then it has probably been too long since you made a commit.

For big changes, it is important that you have helpful and useful descriptions of your work. FreeCAD has adopted a format mentioned in the Pro Git book.

Short (50 chars or less) summary of changes
 
 More detailed explanatory text, if necessary.  Wrap it to about 72
 characters or so.  In some contexts, the first line is treated as the
 subject of an email and the rest of the text as the body.  The blank
 line separating the summary from the body is critical (unless you omit
 the body entirely); tools like rebase can get confused if you run the
 two together.
 
 Further paragraphs come after blank lines. 
 
  - Bullet points are okay, too
 
  - Typically a hyphen or asterisk is used for the bullet, preceded by a
    single space, with blank lines in between, but conventions vary here

If you are doing a lot of related work in a branch, you should make many small commits (see a forum post). When you want to merge those changes into the master branch, you should issue

git log master..myNewBranch

to see the individual commit messages. Then you can write a high quality message when performing a merge.

When you merge to master use the --squash option and commit with your quality commit message. This will allow you to be very liberal with your commits and help to provide a good level of detail in commit messages without so many distinct descriptions.

Pushing your work to your GitHub repository

The local branches in your computer aren't automatically synchronized with the remote servers that you have specified as origin or upstream (see Remote repositories); you have to explicitly push the branches to the remote servers, for which you must have write access. Once you do this, the branches become public, and available for review by other developers.

For FreeCAD, you should push your local branch to the origin remote repository, that is, https://github.com/GITHUB_USERNAME/FreeCAD. You need to enter your username and password every time you push, unless you have set up Credential caching. Please read Pushing commits to a remote repository for more information.

git push origin myNewBranch

Rebasing from upstream

While you work on your own branch, the official FreeCAD code keeps "moving forward" with commits from other developers, and thus starts diverging from the code that you have in your personal fork.

     A----- myNewBranch
     | 
-----o-----------Z FreeCAD master

Therefore, when you are ready to merge your branch to the main FreeCAD repository, you must "rebase" your own copy of the repository, so that it is as close as possible to the official repository. See Git Branching - Rebasing for more information.

git checkout myNewBranch
git pull --rebase upstream master

This will download the code from the master branch of the upstream repository (the official FreeCAD source), and will merge it with your current branch (myNewBranch), that is, your changes will be placed on top of the latest official code. If nobody modified the same files that you did, then the merge will succeed without problems. If some files were changed at the same time by different people, there may be a conflict that needs to be resolved.

                 A'---- myNewBranch
                 |
-----o-----------Z FreeCAD master

To summarize, you need to be in the appropriate branch, rebase the upstream code, and then proceed with the push.

git checkout myNewBranch
git pull --rebase upstream master
git push origin myNewBranch

The pull operation is equivalent to a fetch followed by a merge. When the --rebase option is used, instead of doing a simple merge, it runs the rebase operation.

git pull upstream

git fetch upstream
git merge FETCH_HEAD
git pull --rebase upstream master

git fetch upstream
git rebase master

Merging the branch (pull request)

Once you have committed your changes locally, rebased your branch from the upstream repository, and pushed your branch online, you can initiate a "pull request". A pull request tells the administrators of the official FreeCAD repository that you want to merge the new code in your branch with the official code.

As soon as your push is successful, in your origin repository https://github.com/GITHUB_USERNAME/FreeCAD, GitHub will display a message telling you that a branch was recently pushed, and it will give you the option of comparing and creating the pull request. By pressing Compare & pull request you will find an interface telling you which repository is the "base", target of the merge, and which is the "head", your added branch. A quick check will be done by the system telling you if there are no conflicts with the files that you modified; if you worked on files that nobody has touched, your branch will be able to merge cleanly. In addition, it will show you a text editor so you can write a message documenting your changes; it will also display the number of commits in your branch, the number of files that were modified, and a view showing you the differences between the "base" (official) and the "head" repositories (your own branch) so that everybody can immediately see your intended modifications.

base repository: FreeCAD/FreeCAD    base: master  <----  head repository: GITHUB_USERNAME/FreeCAD    compare: myNewBranch

Able to merge. These branches can be automatically merged.

Click Create pull request to proceed. A message will appear indicating that some checks need to be done on the code. This is a system that compiles FreeCAD automatically and runs the unit tests. If the tests pass, the pull request will have a better chance of being merged into the main code, otherwise a report will be made indicating the errors encountered. See FreeCAD pull requests.

Some checks haven’t completed yet

* continuous-integration/travis-ci/pr Pending — The Travis CI build is in progress

The pull request interface can be used whenever you want to submit code from your own repositories to another repository in GitHub. You can even use it to merge code in the opposite direction, from other people's branches to your own, or between your own branches. In the latter case, since you own the branches, the merges can be approved immediately by yourself.

base repository: SomeProject/Some_Software  base: master       <----  head repository: GITHUB_USERNAME/Some_Software  compare: add_new_functions
base repository: GITHUB_USERNAME/FreeCAD    base: myNewBranch  <----  head repository: FreeCAD/FreeCAD                compare: master
base repository: GITHUB_USERNAME/FreeCAD    base: myNewBranch  <----  head repository: GITHUB_USERNAME/FreeCAD        compare: fix-many-bugs-branch

Keeping the GitHub repository up to date

Once you've forked FreeCAD, your personal repository exists independently from the original. When the original repository has new commits, GitHub will inform you that your personal repository is behind in number of commits:

This branch is 5 commits behind FreeCAD:master.

In similar way, if you created a development branch with new code, GitHub will inform you that this branch is ahead in number of commits; that is, this branch has changes that haven't been merged by the official FreeCAD repository:

This branch is 3 commits ahead of FreeCAD:master.

While developing, both cases are possible, as your own branch may lack commits made by other developers, but include new commits by you:

This branch is 5 commits behind and 3 commits ahead of FreeCAD:master.

In general, rebasing is desirable so you are always ahead of the FreeCAD master code.

Your GitHub repository will never be automatically updated by GitHub; this is something that you must do yourself.

Advanced git operations

Check out GitHub requests locally

Checkout GitHub pull requests locally

Resolving merge conflicts

Creating patches from git

Although Git allows you to merge different branches of code quickly, there are times when it may be desirable to create a patch instead of merging or performing a pull request. The following workflow explains how to do this.

  • You should be developing your new code in a secondary branch of your repository, and not in the master branch. So the first step is to make sure you are in the correct branch.
git branch -v
git checkout myBranch
  • Now use git format-patch against the master branch, and use the --stdout option to redirect the result to standard output; then redirect the standard output to a file, which for convenience is created above the source code directory.
git format-patch master --stdout > ../myCode.patch
  • Another method is
git format-patch HEAD^
git format-patch HEAD~1

The number of circumflex carets ^ or the number 1 indicate the number of commits that should be considered, that is, ^^^ or ~3 will create three patches for three commits.

git format-patch HEAD^

This will create a patch or series of patches with the following naming convention

XXXX-commit-message.patch

where XXXX is a number from 0000 to 9999, and the commit message forms the majority of the file name, for example,

0001-fix-ViewProjMatrix-getProjectionMatrix.patch

Applying patches via git

Git can merge patches or diffs. To know more about this process read Applying patches with Git.

If you already have the patch file in your system, just apply it.

git apply myCode.patch

You can use curl to download a patch from a website, and then apply it through git.

curl -O https://some.website.org/code/myCode.patch
git apply myCode.patch

Add .diff or .patch at the end of the URL of a GitHub commit, pull request, or compare view so that the website shows you the plain text view of that page.

You can point curl to a particular commit patch in the repository, and pipe it directly to git to apply the patch.

curl https://github.com/FreeCAD/FreeCAD/commit/c476589652a0f67b544735740e20ff702e8d0621.patch | git apply -

Reversing a patch in git

When you apply a patch you modify some files. However, these modifications aren't permanent until you commit the changes. Therefore, if you want to revert a patch use the following instructions.

This will revert the changes applied, if you still have access to the original patch file.

git apply -R myCode.patch

Alternatively, this will remove non-committed changes to the branch.

git checkout -f

Stashing git commits

Say that you're working on a branch and you find yourself making some modifications to the source that are out of the scope of your current branch; in other words, those changes would be better in another branch instead of the current one. The git stash command can be used to temporarily store those uncommitted local changes.

git stash

If in the future you want to use those commits, you can apply the stash.

git stash apply

Or if you decide that you don't like the commits anymore, you may delete the stash.

git stash pop

You can list multiple stashes with

git stash list

To learn more, read Useful tricks you might not know about Git stash.

FreeCAD revision number

In contrast to subversion, which uses a consecutive number for its revisions, Git produces SHA-1 hash values with every commit. A hash value is a long alphanumeric string that looks like this

9b3ffef570596e184006287434fba54a4b03ccc3

Latest revision number

To find the latest revision number of a particular branch use the rev-list operation with the --count option. Give the name of the branch, remote repository, tag, or a special pointer like HEAD, to indicate the last commit in that particular object.

git rev-list --count master
git rev-list --count HEAD
git rev-list --count origin

Or browse the repository on GitHub, and read the amount of commits reported in the particular branch.

Revision number of a specific commit hash

Since the hash is an alphanumeric string it is not very useful to decide if a certain commit is older or newer than another hash. To find the revision number of a particular hash, again use the rev-list operation; the input can be the full hash, or a partial hash that is unique, usually the first 7 digits are enough.

git rev-list --count ab1520b872821414c6ce4a15fb85d471ac2a2b03
git rev-list --count 9948ee4

Revision hash of a specific commit number

If we have the commit number, say, 15000, and we want to find the corresponding hash, we need to calculate the number of commits since this point until the last commit (HEAD). First, get the latest commit number.

git rev-list --count HEAD
17465

Then subtract the commit that we want.

17465 - 15000 = 2465

Then use the log operation to show all commits and hashes. The --skip option jumps the difference in commits that we calculated so that we go directly to the hash that we are looking for.

git log --skip=2465
commit 44c2f19e380e76b567d114a6360519d66f7a9e24

Confirm it's the right commit number.

git rev-list --count 44c2f19e38
15000

Revision number in FreeCAD's interface

The version number that appears in Help → About FreeCAD → Copy to clipboard is defined in src/Build/Version.h, which is created at compile time when the cmake tool is run. Read Extract version number from git source for more information.

See the compile on Unix page for the full information on compiling FreeCAD.

Alternative repositories

The beauty of git is that everybody can clone a project, and start modifying the code. Several frequent collaborators of the FreeCAD project have their own git repository, where they build up their work before it is ready to be included in the official source code, or simply where they experiment new ideas. In certain cases, you might want to clone your FreeCAD code from one of these, instead of the official repos, to benefit from the changes their users did.

Be warned, though, that this is at your own risk, and only the official repository above is fully guaranteed to work and contain clean code.

It is also possible to attach several remote repositories to a same local FreeCAD git code, using the "git remote" command. This is useful to keep in sync with the master code branch, but keep an eye on the work of different developers.

Further reading