Difference between revisions of "Use uGFX repository as Git Submodule"

From uGFX Wiki
Jump to: navigation, search
(The Problem)
 
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
The following article is a guide on how to add the uGFX repository as a submodule in an existing git repository. This guide was written by one of our users. Make sure that you check out his tech [http://wolinlabs.com/blog/ blog].
+
The following article is a guide on how to add the uGFX repository as a submodule in an existing git repository. This guide was written by one of our users. Check out his tech [http://wolinlabs.com/blog/ blog].
  
 
== The Problem ==
 
== The Problem ==
I am writing firmware for  an embedded system which uses the uGFX graphics library.  I use Git for source control, and wanted to use a Git submodule  to embed uGFX's Bitbucket hosted repository into my source repository.  Additionally, I wanted to be able to add some custom board and driver files to uGFX which are needed for my project, but are of no value to uGFX (and won't be pushed back to their repository.)  
+
I am writing firmware for  an embedded system which uses the uGFX graphics library.  I use Git for source control, and wanted to use a Git submodule  to embed uGFX's repository into my source repository.  Additionally, I wanted to be able to add some custom board and driver files to uGFX which are needed for my project, but are of no value to uGFX (and won't be pushed back to their repository).
  
 
Initially, I merely added a Git submodule to my local repository, made some changes and commited them to the submodule's master branch.  This worked smashingly... until I tried to make a local clone of my repository, using the recursive switch:
 
Initially, I merely added a Git submodule to my local repository, made some changes and commited them to the submodule's master branch.  This worked smashingly... until I tried to make a local clone of my repository, using the recursive switch:
  
<source lang="text">
+
<syntaxhighlight lang=text>
 
git clone --recursive myrepo myrepo_clone
 
git clone --recursive myrepo myrepo_clone
</source>
+
</syntaxhighlight>
  
 
and got an error:
 
and got an error:
<br>
+
<syntaxhighlight lang=text>
<i>fatal: reference is not a tree: ca025c09ca2b7b82ca086309eeb0696d674cb1d0 Unable to checkout 'ca025c09ca2b7b82ca086309eeb0696d674cb1d0' in submodule path 'ugfx_submodule'</i>
+
fatal: reference is not a tree: ca025c09ca2b7b82ca086309eeb0696d674cb1d0
 
+
Unable to checkout 'ca025c09ca2b7b82ca086309eeb0696d674cb1d0' in submodule path 'ugfx_submodule'
 +
</syntaxhighlight>
  
 
The problem is that my changes to the submodule's master branch are in my local repository, but when Git tries to clone the uGFX submodule, it gets source from the Bitbucket repository, which obviously does not have my changes.
 
The problem is that my changes to the submodule's master branch are in my local repository, but when Git tries to clone the uGFX submodule, it gets source from the Bitbucket repository, which obviously does not have my changes.
  
 +
What I'm doing seems to me like a pretty common situation/workflow. Surprisingly, when I searched, I did not find  documentation/examples about how to set this up properly so that recursive clones would work, etc.
  
What I'm doing  seems to me like a pretty common situation/workflow...  surprisingly, when I searched, I did not find  documentation/examples about how to set this up properly so that recursive clones would work, etc.
+
'''Here's  my requirements in more detail:'''
 
+
# I need to get code from the uGFX Bitbucket repository's master branch, pull in changes from that branch when available, and integrate them into my product after testing.
 
+
<b>Here's  my requirements in more detail:</b>
+
 
+
# I need to get code from thce uGFX Bitbucket repository's master branch, pull in changes from that branch when available, and integrate them into my product after testing.
+
 
# I need to be able to add code to the local uGFX tree and keep these additions under source control, but not push them back to the uGFX repository (my code is specific to my project and proprietary h/w, trust me, uGFX doesn't  want it. =)
 
# I need to be able to add code to the local uGFX tree and keep these additions under source control, but not push them back to the uGFX repository (my code is specific to my project and proprietary h/w, trust me, uGFX doesn't  want it. =)
# Other users must be able to clone my repository. They don't have to be able to update the uGFX tree with the Bitbucket changes (I will handle when those changes are integrated) --- although they could if they made minor modifications to their repository.
+
# Other users must be able to clone my repository. They don't have to be able to update the uGFX tree with the Bitbucket changes (I will handle when those changes are integrated) - although they could if they made minor modifications to their repository.
  
 
== My Solution ==
 
== My Solution ==
<p>I came up  with a workable solution,  involving a version of  
+
I came up  with a workable solution,  involving a version of  
[https://felipec.wordpress.com/2014/05/11/git-triangular-workflows Triangular Workflow],  
+
[https://felipec.wordpress.com/2014/05/11/git-triangular-workflows Triangular Workflow], using an upstream repository. I used a Raspberry Pi + Raspbian for the upstream repository because I had one on my network running DNS/DHCP and thought it would be cool, you could just as easily use a second local repository on your disk, Bitbucket/Github, etc.
using an upstream repository.   I used a Raspberry Pi + Raspbian for the upstream repository because I had one on my network running DNS/DHCP and thought it would be cool, you could just as easily use a second local repository on your disk, Bitbucket/Github, etc, etc</p>
+
  
<b>Here are the specifics for my solution:</b>
+
'''Here are the specifics for my solution:'''
<ul>
+
* Create an upstream Git server. I used a Raspberry Pi and put the repos on a USB stick, because it was convenient, starting with [http://www.instructables.com/id/GitPi-A-Private-Git-Server-on-Raspberry-Pi/?ALLSTEPS this article]. I added some enhancements from [https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server the Git book], like adding a git user and some authorized public ssh keys, etc...
<li>Create an upstream Git server. I used a Raspberry Pi and put the repos on a USB stick, because it was convenient, starting with  
+
[http://www.instructables.com/id/GitPi-A-Private-Git-Server-on-Raspberry-Pi/?ALLSTEPS this article]
+
.   I added some enhancements from [https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server the Git book], like adding a git user and some authorized public ssh keys, etc...</li>
+
  
  
<li>Make two bare repositories on the Git server, for this example call them myproduct.git and myproduct_ugfx.git</li>
+
* Make two bare repositories on the Git server, for this example call them myproduct.git and myproduct_ugfx.git
  
  
<li>Push my repository's master branch to the myproduct.git repository. I didn't write this part down, but I believe I just added a remote for origin and pushed the master branch:</li>
+
* Push my repository's master branch to the myproduct.git repository. I didn't write this part down, but I believe I just added a remote for origin and pushed the master branch:
 
+
<syntaxhighlight lang=text>
<source lang="text">
+
 
git remote add origin git@pi-b:/mnt/git_repos/myproduct.git
 
git remote add origin git@pi-b:/mnt/git_repos/myproduct.git
 
git push
 
git push
</source>
+
</syntaxhighlight>
  
  
<li>From the top directory of my repository, I add the uGFX submodule</li>
+
* From the top directory of my repository, I add the uGFX submodule
 +
<syntaxhighlight lang=text>
 +
git submodule add https://git.ugfx.io/ugfx/ugfx.git ugfx_submodule
 +
</syntaxhighlight>
  
<source lang="text">
 
git submodule add https://bitbucket.org/Tectu/ugfx.git ugfx_submodule
 
</source>
 
  
 
+
* cd into the submodule's directory, and change the name of the submodule's master branch to updates
<li>cd into the submodule's directory, and change the name of the submodule's master branch to updates</li>
+
<syntaxhighlight lang=text>
 
+
<source lang="text">
+
 
git branch -m master updates
 
git branch -m master updates
</source>
+
</syntaxhighlight>
 
+
  
<li>Rename the remote origin to bitbucket, which the updates (formerly master) branch tracks</li>
 
  
 +
* Rename the remote origin to bitbucket, which the updates (formerly master) branch tracks
 
<source lang="text">
 
<source lang="text">
 
git remote rename origin bitbucket
 
git remote rename origin bitbucket
Line 71: Line 61:
  
  
<li>At this point, run <span class="code">git branch -vv</span> as a check, and it should show one branch named updates which tracks remote bitbucket's master branch</li>
+
* At this point, run <span class="code">git branch -vv</span> as a check, and it should show one branch named updates which tracks remote bitbucket's master branch
  
  
<li>The submodule no longer has a master branch or origin. I wanted the submodule master branch to track my Pi Git server's uGFX repository, so I add a remote for origin, create a master branch, and push the master branch to the server (telling it to track that branch):</li>
+
* The submodule no longer has a master branch or origin. I wanted the submodule master branch to track my Pi Git server's uGFX repository, so I add a remote for origin, create a master branch, and push the master branch to the server (telling it to track that branch):
 
+
<syntaxhighlight lang=text>
<source lang="text">
+
 
git remote add origin git@pi-b:/mnt/git_repos/myproduct_ugfx.git
 
git remote add origin git@pi-b:/mnt/git_repos/myproduct_ugfx.git
 
git checkout -b master  
 
git checkout -b master  
git push -u origin master</span>
+
git push -u origin master
</source>
+
</syntaxhighlight>
  
  
<li>Now that I've pushed the submodule to my server, I change the uGFX submodule to point to my server, so when people clone the whole repo with --recursive, they get  my master branch with my additions</li>
+
* Now that I've pushed the submodule to my server, I change the uGFX submodule to point to my server, so when people clone the whole repo with --recursive, they get  my master branch with my additions
 
+
<syntaxhighlight lang=text>
<source lang="text">
+
 
git config remote.origin.url git@pi-b:/mnt/git_repos/myproduct_ugfx.git
 
git config remote.origin.url git@pi-b:/mnt/git_repos/myproduct_ugfx.git
</source>
+
</syntaxhighlight>
</ul>
+
At this point, I wanted to add a few files to the uGFX tree. I add them to the master branch, commit, and push (which goes to the Pi repository.) Remember to also go up a direcotry, outside of the submodule, run <span class="code">git submodule sync</span> (not sure if that's necessary), and then commit+push those changes to the main/outer repo.
  
 +
Now, when there are changes on the Bitbucket repository, in the submodule I can pull them into my local updates branch, then when I've tested them and want to integrate them into the my product, I can merge updates with master, then commit and push master to the Pi repository.
  
  
 
+
Local recursive clones (like the one above that caused the initial error) will now work. If someone wants to clone the myproduct repository from the Pi, provided they have access to the Pi (see Git book for adding an ssh key) they can do it with
<p>At this point, I wanted to add a few files to the uGFX tree. I add them to the master branch, commit, and push (which goes to the Pi repository.) Remember to also go up a direcotry, outside of the submodule, run <span class="code">git submodule sync</span> (not sure if that's necessary), and then commit+push those changes to the main/outer repo.</p>
+
<syntaxhighlight lang=text>
 
+
 
+
<p>Now, when there are changes on the Bitbucket repository, in the submodule I can pull them into my local updates branch, then when I've tested them and want to integrate them into the my product, I can merge updates with master, then commit and push master to the Pi repository.</p>
+
 
+
 
+
<p>Local recursive clones (like the one above that caused the initial error) will now work. If someone wants to clone the myproduct repository from the Pi, provided they have access to the Pi (see Git book for adding an ssh key) they can do it with</p>
+
 
+
<source lang="text">
+
 
git clone --recursive git@pi-b:/mnt/git_repos/myproduct.git
 
git clone --recursive git@pi-b:/mnt/git_repos/myproduct.git
</source>
+
</syntaxhighlight>
 
+
  
 
== Links ==
 
== Links ==
 
* [https://felipec.wordpress.com/2014/05/11/git-triangular-workflows Triangular workflow example]
 
* [https://felipec.wordpress.com/2014/05/11/git-triangular-workflows Triangular workflow example]
* [http://www.instructables.com/id/GitPi-A-Private-Git-Server-on-Raspberry-Pi/?ALLSTEPS Simple RPi Git Server]
 
* [https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server Git book, server chapter]
 
 
* [http://wolinlabs.com/blog Ross Wolin's Tech Blog]
 
* [http://wolinlabs.com/blog Ross Wolin's Tech Blog]

Latest revision as of 16:59, 3 August 2021

The following article is a guide on how to add the uGFX repository as a submodule in an existing git repository. This guide was written by one of our users. Check out his tech blog.

The Problem

I am writing firmware for an embedded system which uses the uGFX graphics library. I use Git for source control, and wanted to use a Git submodule to embed uGFX's repository into my source repository. Additionally, I wanted to be able to add some custom board and driver files to uGFX which are needed for my project, but are of no value to uGFX (and won't be pushed back to their repository).

Initially, I merely added a Git submodule to my local repository, made some changes and commited them to the submodule's master branch. This worked smashingly... until I tried to make a local clone of my repository, using the recursive switch:

git clone --recursive myrepo myrepo_clone

and got an error:

fatal: reference is not a tree: ca025c09ca2b7b82ca086309eeb0696d674cb1d0
Unable to checkout 'ca025c09ca2b7b82ca086309eeb0696d674cb1d0' in submodule path 'ugfx_submodule'

The problem is that my changes to the submodule's master branch are in my local repository, but when Git tries to clone the uGFX submodule, it gets source from the Bitbucket repository, which obviously does not have my changes.

What I'm doing seems to me like a pretty common situation/workflow. Surprisingly, when I searched, I did not find documentation/examples about how to set this up properly so that recursive clones would work, etc.

Here's my requirements in more detail:

  1. I need to get code from the uGFX Bitbucket repository's master branch, pull in changes from that branch when available, and integrate them into my product after testing.
  2. I need to be able to add code to the local uGFX tree and keep these additions under source control, but not push them back to the uGFX repository (my code is specific to my project and proprietary h/w, trust me, uGFX doesn't want it. =)
  3. Other users must be able to clone my repository. They don't have to be able to update the uGFX tree with the Bitbucket changes (I will handle when those changes are integrated) - although they could if they made minor modifications to their repository.

My Solution

I came up with a workable solution, involving a version of Triangular Workflow, using an upstream repository. I used a Raspberry Pi + Raspbian for the upstream repository because I had one on my network running DNS/DHCP and thought it would be cool, you could just as easily use a second local repository on your disk, Bitbucket/Github, etc.

Here are the specifics for my solution:

  • Create an upstream Git server. I used a Raspberry Pi and put the repos on a USB stick, because it was convenient, starting with this article. I added some enhancements from the Git book, like adding a git user and some authorized public ssh keys, etc...


  • Make two bare repositories on the Git server, for this example call them myproduct.git and myproduct_ugfx.git


  • Push my repository's master branch to the myproduct.git repository. I didn't write this part down, but I believe I just added a remote for origin and pushed the master branch:
git remote add origin git@pi-b:/mnt/git_repos/myproduct.git
git push


  • From the top directory of my repository, I add the uGFX submodule
git submodule add https://git.ugfx.io/ugfx/ugfx.git ugfx_submodule


  • cd into the submodule's directory, and change the name of the submodule's master branch to updates
git branch -m master updates


  • Rename the remote origin to bitbucket, which the updates (formerly master) branch tracks
git remote rename origin bitbucket


  • At this point, run git branch -vv as a check, and it should show one branch named updates which tracks remote bitbucket's master branch


  • The submodule no longer has a master branch or origin. I wanted the submodule master branch to track my Pi Git server's uGFX repository, so I add a remote for origin, create a master branch, and push the master branch to the server (telling it to track that branch):
git remote add origin git@pi-b:/mnt/git_repos/myproduct_ugfx.git
git checkout -b master 
git push -u origin master


  • Now that I've pushed the submodule to my server, I change the uGFX submodule to point to my server, so when people clone the whole repo with --recursive, they get my master branch with my additions
git config remote.origin.url git@pi-b:/mnt/git_repos/myproduct_ugfx.git

At this point, I wanted to add a few files to the uGFX tree. I add them to the master branch, commit, and push (which goes to the Pi repository.) Remember to also go up a direcotry, outside of the submodule, run git submodule sync (not sure if that's necessary), and then commit+push those changes to the main/outer repo.

Now, when there are changes on the Bitbucket repository, in the submodule I can pull them into my local updates branch, then when I've tested them and want to integrate them into the my product, I can merge updates with master, then commit and push master to the Pi repository.


Local recursive clones (like the one above that caused the initial error) will now work. If someone wants to clone the myproduct repository from the Pi, provided they have access to the Pi (see Git book for adding an ssh key) they can do it with

git clone --recursive git@pi-b:/mnt/git_repos/myproduct.git

Links