Ben Chuanlong Du's Blog

It is never too late to learn.

Hands on Dulwich

Note: dulwich is not feature complete yet and the development of the project is extremely slow. It is suggested that you use other Python packages instead. For more discussions, please refer to Git Implementations and Bindings in Python .

Tips and Traps

  1. The git command (and thus Dulwich) accepts URLs both with and without the trailing .git.
In [1]:
!pip3 install dulwich
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: dulwich in /usr/local/lib/python3.10/dist-packages (0.21.5)
Requirement already satisfied: urllib3>=1.25 in /usr/local/lib/python3.10/dist-packages (from dulwich) (2.0.3)

[notice] A new release of pip is available: 23.2 -> 23.2.1
[notice] To update, run: python3 -m pip install --upgrade pip
In [30]:
import dulwich.repo
In [31]:
from dulwich.repo import Repo
In [32]:
import dulwich.porcelain
In [6]:
url = "https://github.com/dclong/test_dulwich"
dir_local = "/tmp/test_dulwich"
!rm -rf {dir_local}

git clone

In [7]:
repo = dulwich.porcelain.clone(url, dir_local)
In [77]:
repo
Out[77]:
<Repo at '/tmp/test_dulwich'>
In [9]:
!ls {dir_local}
abc  build.sh  readme.md

Clone the local repository to another location (which is not very useful as you can directly copy the directory to the new location).

In [ ]:
?repo.clone
Signature:
repo.clone(
    target_path,
    *,
    mkdir=True,
    bare=False,
    origin=b'origin',
    checkout=None,
    branch=None,
    progress=None,
    depth=None,
    symlinks=None,
) -> 'Repo'
Docstring:
Clone this repository.

Args:
  target_path: Target path
  mkdir: Create the target directory
  bare: Whether to create a bare repository
  checkout: Whether or not to check-out HEAD after cloning
  origin: Base name for refs in target repository
    cloned from this repository
  branch: Optional branch or tag to be used as HEAD in the new repository
    instead of this repository's HEAD.
  progress: Optional progress function
  depth: Depth at which to fetch
  symlinks: Symlinks setting (default to autodetect)
Returns: Created repository as `Repo`
File:      /usr/local/lib/python3.10/dist-packages/dulwich/repo.py
Type:      method

dulwich.repo.Repo

In [20]:
type(repo)
Out[20]:
dulwich.repo.Repo

Create a repo from a local directory.

In [11]:
repo2 = Repo("/tmp/test_dulwich")
repo2
Out[11]:
<Repo at '/tmp/test_dulwich'>
In [21]:
dir(repo)
Out[21]:
['__class__',
 '__contains__',
 '__delattr__',
 '__delitem__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__enter__',
 '__eq__',
 '__exit__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getitem__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__setitem__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 '_add_graftpoints',
 '_commondir',
 '_controldir',
 '_del_named_file',
 '_determine_file_mode',
 '_get_object',
 '_get_user_identity',
 '_graftpoints',
 '_init_files',
 '_init_maybe_bare',
 '_init_new_working_directory',
 '_put_named_file',
 '_read_heads',
 '_remove_graftpoints',
 '_write_reflog',
 'bare',
 'clone',
 'close',
 'commondir',
 'controldir',
 'create',
 'discover',
 'do_commit',
 'fetch',
 'fetch_objects',
 'fetch_pack_data',
 'generate_pack_data',
 'get_blob_normalizer',
 'get_config',
 'get_config_stack',
 'get_description',
 'get_graph_walker',
 'get_named_file',
 'get_object',
 'get_parents',
 'get_peeled',
 'get_refs',
 'get_shallow',
 'get_walker',
 'has_index',
 'head',
 'hooks',
 'index_path',
 'init',
 'init_bare',
 'object_store',
 'open_index',
 'parents_provider',
 'path',
 'refs',
 'reset_index',
 'set_description',
 'stage',
 'update_shallow']

git status

In [44]:
s = dulwich.porcelain.status(repo)
dir(s)
Out[44]:
['__add__',
 '__class__',
 '__class_getitem__',
 '__contains__',
 '__delattr__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getitem__',
 '__getnewargs__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__iter__',
 '__le__',
 '__len__',
 '__lt__',
 '__match_args__',
 '__module__',
 '__mul__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__rmul__',
 '__setattr__',
 '__sizeof__',
 '__slots__',
 '__str__',
 '__subclasshook__',
 '_asdict',
 '_field_defaults',
 '_fields',
 '_make',
 '_replace',
 'count',
 'index',
 'staged',
 'unstaged',
 'untracked']
In [47]:
s.staged
Out[47]:
{'add': [], 'delete': [], 'modify': []}
In [48]:
s.unstaged
Out[48]:
[]
In [49]:
s.untracked
Out[49]:
[]
In [76]:
!git -C {dir_local} status
On branch main
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   test1.txt

no changes added to commit (use "git add" and/or "git commit -a")
In [78]:
dulwich.porcelain.get_tree_changes(repo)
Out[78]:
{'add': [], 'delete': [], 'modify': []}
In [56]:
!git -C {dir_local} diff
diff --git a/test1.txt b/test1.txt
index 9daeafb..a5f96b1 100644
--- a/test1.txt
+++ b/test1.txt
@@ -1 +1,3 @@
 test
+add a new line
+
In [64]:
repo[b"head"]
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[64], line 1
----> 1 repo[b"head"]

File /usr/local/lib/python3.10/dist-packages/dulwich/repo.py:783, in BaseRepo.__getitem__(self, name)
    781         pass
    782 try:
--> 783     return self.object_store[self.refs[name]]
    784 except RefFormatError as exc:
    785     raise KeyError(name) from exc

File /usr/local/lib/python3.10/dist-packages/dulwich/refs.py:326, in RefsContainer.__getitem__(self, name)
    324 _, sha = self.follow(name)
    325 if sha is None:
--> 326     raise KeyError(name)
    327 return sha

KeyError: b'head'
In [65]:
dir(repo)
Out[65]:
['__annotations__',
 '__class__',
 '__contains__',
 '__delattr__',
 '__delitem__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__enter__',
 '__eq__',
 '__exit__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getitem__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__setitem__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 '_add_graftpoints',
 '_commondir',
 '_controldir',
 '_del_named_file',
 '_determine_file_mode',
 '_determine_symlinks',
 '_get_object',
 '_get_user_identity',
 '_graftpoints',
 '_init_files',
 '_init_maybe_bare',
 '_init_new_working_directory',
 '_put_named_file',
 '_read_heads',
 '_remove_graftpoints',
 '_write_reflog',
 'bare',
 'clone',
 'close',
 'commondir',
 'controldir',
 'create',
 'discover',
 'do_commit',
 'fetch',
 'fetch_pack_data',
 'find_missing_objects',
 'generate_pack_data',
 'get_blob_normalizer',
 'get_config',
 'get_config_stack',
 'get_description',
 'get_graph_walker',
 'get_named_file',
 'get_object',
 'get_parents',
 'get_peeled',
 'get_refs',
 'get_shallow',
 'get_walker',
 'get_worktree_config',
 'has_index',
 'head',
 'hooks',
 'index_path',
 'init',
 'init_bare',
 'object_store',
 'open_index',
 'parents_provider',
 'path',
 'refs',
 'reset_index',
 'set_description',
 'stage',
 'unstage',
 'update_shallow']
In [66]:
repo.head
Out[66]:
<bound method BaseRepo.head of <Repo at '/tmp/test_dulwich'>>
In [68]:
repo.head()
Out[68]:
b'729bb376c018f068548c574a9fa05764432ef33a'
In [72]:
repo[repo.head()]
Out[72]:
<Commit b'729bb376c018f068548c574a9fa05764432ef33a'>
In [73]:
repo.object_store
Out[73]:
<DiskObjectStore('/tmp/test_dulwich/.git/objects')>
In [74]:
import sys

from dulwich.patch import write_tree_diff

outstream = getattr(sys.stdout, 'buffer', sys.stdout)
write_tree_diff(outstream, repo.object_store, repo[repo.head()], commit.tree)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[74], line 6
      3 from dulwich.patch import write_tree_diff
      5 outstream = getattr(sys.stdout, 'buffer', sys.stdout)
----> 6 write_tree_diff(outstream, repo.object_store, repo[repo.head()], commit.tree)

NameError: name 'commit' is not defined
In [ ]:
import sys

from dulwich.patch import write_tree_diff
from dulwich.repo import Repo

repo_path = "."
commit_id = b"a6602654997420bcfd0bee2a0563d9416afe34b4"

r = Repo(repo_path)

commit = r[commit_id]
parent_commit = r[commit.parents[0]]
outstream = getattr(sys.stdout, 'buffer', sys.stdout)
write_tree_diff(outstream, r.object_store, parent_commit.tree, commit.tree)

git add

In [33]:
!touch /tmp/test_dulwich/abc2
In [34]:
!ls /tmp/test_dulwich/
abc  abc2  build.sh  Dockerfile  LICENSE  readme.md  scripts
In [35]:
!git -C /tmp/test_dulwich/ status
On branch dev
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   abc

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	abc2

In [29]:
?dulwich.porcelain.add
Signature: dulwich.porcelain.add(repo='.', paths=None)
Docstring:
Add files to the staging area.

Args:
  repo: Repository for the files
  paths: Paths to add.  No value passed stages all modified files.
Returns: Tuple with set of added files and ignored files

If the repository contains ignored directories, the returned set will
contain the path to an ignored directory (with trailing slash). Individual
files within ignored directories will not be returned.
File:      ~/.local/lib/python3.8/site-packages/dulwich/porcelain.py
Type:      function
In [32]:
dulwich.porcelain.add("/tmp/test_dulwich", paths="/tmp/test_dulwich/abc")
Out[32]:
(['abc'], set())

By default, dulwich adds all files in the current working directory, which is not the right behavior! I have submitted a ticket to fix the issue.

In [54]:
dulwich.porcelain.add("/tmp/test_dulwich")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_300/40685537.py in <module>
----> 1 dulwich.porcelain.add("/tmp/test_dulwich")

~/.local/lib/python3.8/site-packages/dulwich/porcelain.py in add(repo, paths)
    512         ignore_manager = IgnoreFilterManager.from_repo(r)
    513         if not paths:
--> 514             paths = list(
    515                 get_untracked_paths(
    516                     str(Path(os.getcwd()).resolve()),

~/.local/lib/python3.8/site-packages/dulwich/porcelain.py in get_untracked_paths(frompath, basepath, index, exclude_ignored)
   1291     ):
   1292         if not is_dir:
-> 1293             ip = path_to_tree_path(basepath, ap)
   1294             if ip not in index:
   1295                 if (

~/.local/lib/python3.8/site-packages/dulwich/porcelain.py in path_to_tree_path(repopath, path, tree_encoding)
    252 
    253         try:
--> 254             relpath = resolved_path.relative_to(repopath)
    255         except ValueError:
    256             # If path is a symlink that points to a file outside the repo, we

/usr/lib/python3.8/pathlib.py in relative_to(self, *other)
    906         if (root or drv) if n == 0 else cf(abs_parts[:n]) != cf(to_abs_parts):
    907             formatted = self._format_parsed_parts(to_drv, to_root, to_parts)
--> 908             raise ValueError("{!r} does not start with {!r}"
    909                              .format(str(self), str(formatted)))
    910         return self._from_parsed_parts('', root if n == 1 else '',

ValueError: '/workdir/archives/blog/misc/content/2020/11/hands-on-dulwich/hands-on-dulwich.ipynb' does not start with '/tmp/test_dulwich'
In [67]:
dulwich.porcelain.add("/tmp/test_dulwich", paths="/tmp/test_dulwich/.")
Out[67]:
(['./'], set())
In [71]:
status = dulwich.porcelain.status("/tmp/test_dulwich")
status
Out[71]:
GitStatus(staged={'add': [], 'delete': [], 'modify': []}, unstaged=[], untracked=['abc2'])
In [72]:
status.unstaged
Out[72]:
[]
In [73]:
status.untracked
Out[73]:
['abc2']

git commit

In [48]:
?dulwich.porcelain.commit
Signature:
dulwich.porcelain.commit(
    repo='.',
    message=None,
    author=None,
    committer=None,
    encoding=None,
    no_verify=False,
)
Docstring:
Create a new commit.

Args:
  repo: Path to repository
  message: Optional commit message
  author: Optional author name and email
  committer: Optional committer name and email
  no_verify: Skip pre-commit and commit-msg hooks
Returns: SHA1 of the new commit
File:      ~/.local/lib/python3.8/site-packages/dulwich/porcelain.py
Type:      function
In [50]:
dulwich.porcelain.commit("/tmp/test_dulwich/", message="add abc")
Out[50]:
b'11fb9f18f9d211c93175e898faa731584b8be368'
In [51]:
!git -C /tmp/test_dulwich/ status
On branch dev
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	abc2

nothing added to commit but untracked files present (use "git add" to track)
In [53]:
!git -C /tmp/test_dulwich/ log | head
commit 11fb9f18f9d211c93175e898faa731584b8be368
Author: Benjamin Du <longendu@yahoo.com>
Date:   Sat Aug 7 01:33:45 2021 +0000

    add abc

commit 2fd55f0a653bf8ec2e7ffda16c7b2d601167da06
Author: Ben Chuanlong Du <dclong@users.noreply.github.com>
Date:   Fri Jul 16 09:53:00 2021 -0700

ConfigFile

ConfigFile inherits ConfigDict which means that you operate on a ConfiFile like a dict.

In [14]:
config = repo.get_config()
config
Out[14]:
ConfigFile(CaseInsensitiveDict([((b'core',), CaseInsensitiveDict([(b'repositoryformatversion', b'0'), (b'filemode', b'true'), (b'bare', b'false'), (b'logallrefupdates', b'true')])), ((b'remote', b'origin'), CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'), (b'fetch', b'+refs/heads/*:refs/remotes/origin/*')])), ((b'remote', b'nima'), CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'), (b'fetch', b'+refs/heads/*:refs/remotes/nima/*')]))]))
In [37]:
config.keys()
Out[37]:
KeysView(ConfigFile(CaseInsensitiveDict([((b'core',), CaseInsensitiveDict([(b'repositoryformatversion', b'0'), (b'filemode', b'true'), (b'bare', b'false'), (b'logallrefupdates', b'true')])), ((b'remote', b'origin'), CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'), (b'fetch', b'+refs/heads/*:refs/remotes/origin/*')])), ((b'remote', b'nima'), CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'), (b'fetch', b'+refs/heads/*:refs/remotes/nima/*')]))])))
In [38]:
config.values()
Out[38]:
ValuesView(ConfigFile(CaseInsensitiveDict([((b'core',), CaseInsensitiveDict([(b'repositoryformatversion', b'0'), (b'filemode', b'true'), (b'bare', b'false'), (b'logallrefupdates', b'true')])), ((b'remote', b'origin'), CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'), (b'fetch', b'+refs/heads/*:refs/remotes/origin/*')])), ((b'remote', b'nima'), CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'), (b'fetch', b'+refs/heads/*:refs/remotes/nima/*')]))])))
In [40]:
config[(b"remote", b"origin")]
Out[40]:
CaseInsensitiveDict([(b'url', b'https://github.com/dclong/docker-ubuntu_b'),
                     (b'fetch', b'+refs/heads/*:refs/remotes/origin/*')])
In [27]:
config.get((b"remote", b"origin"), b"url")
Out[27]:
b'https://github.com/dclong/docker-ubuntu_b'
In [31]:
dict(config)
Out[31]:
{(b'core',): CaseInsensitiveDict([(b'repositoryformatversion', b'0'),
                      (b'filemode', b'true'),
                      (b'bare', b'false'),
                      (b'logallrefupdates', b'true')]),
 (b'remote',
  b'origin'): CaseInsensitiveDict([(b'url',
                       b'https://github.com/dclong/docker-ubuntu_b'),
                      (b'fetch', b'+refs/heads/*:refs/remotes/origin/*')]),
 (b'remote',
  b'nima'): CaseInsensitiveDict([(b'url',
                       b'https://github.com/dclong/docker-ubuntu_b'),
                      (b'fetch', b'+refs/heads/*:refs/remotes/nima/*')])}

git remote -v

In [31]:
[m for m in dir(dulwich.porcelain) if 'remote' in m]
Out[31]:
['_import_remote_refs',
 'get_branch_remote',
 'get_remote_repo',
 'ls_remote',
 'remote_add',
 'remote_remove']
In [34]:
dulwich.porcelain.get_branch_remote(repo)
Out[34]:
b'origin'
In [36]:
dulwich.porcelain.get_remote_repo(repo)
Out[36]:
('origin', 'https://github.com/dclong/docker-ubuntu_b')
In [39]:
dulwich.porcelain.ls_remote(url)
Out[39]:
{b'HEAD': b'4996e93a5f24c375b1d56deddda1cd9cfddd14f6',
 b'refs/heads/centos7': b'd7a6f672771be1fc33ddd1006b3318901c914b19',
 b'refs/heads/debian': b'c7fc15b52f4c76faa4ba487a113b5c987ac3b371',
 b'refs/heads/dev': b'4996e93a5f24c375b1d56deddda1cd9cfddd14f6',
 b'refs/heads/main': b'925dd68d39ea943f1c387e4906e72aebc765c4bf',
 b'refs/pull/1/head': b'b90bd029b4707a94640957cbfae73a631a9d83e0',
 b'refs/pull/1/merge': b'c767a6a929e599c245a4241477093554c1ab0d10',
 b'refs/pull/10/head': b'ea5b60d7fc34fce65ec4a503c20536d3e0d4587a',
 b'refs/pull/100/head': b'b514f4d74dd93b237cb72a91e992a0480393b546',
 b'refs/pull/101/head': b'b514f4d74dd93b237cb72a91e992a0480393b546',
 b'refs/pull/102/head': b'f2f28a5809657930caa51d2606dc62c3e74d2e27',
 b'refs/pull/103/head': b'f2f28a5809657930caa51d2606dc62c3e74d2e27',
 b'refs/pull/104/head': b'43863e528cbd069b8f095f8eade79ae990fbe916',
 b'refs/pull/105/head': b'43863e528cbd069b8f095f8eade79ae990fbe916',
 b'refs/pull/106/head': b'238b7db8c24f6ed43125696405f56cbd33302ad1',
 b'refs/pull/107/head': b'238b7db8c24f6ed43125696405f56cbd33302ad1',
 b'refs/pull/108/head': b'2e5bec99f5b0bfcb5e74ad8b9c0ff4c36827a295',
 b'refs/pull/109/head': b'4996e93a5f24c375b1d56deddda1cd9cfddd14f6',
 b'refs/pull/11/head': b'0d7e63476077d2ed51728823f3ce54b57a8287e6',
 b'refs/pull/110/head': b'4996e93a5f24c375b1d56deddda1cd9cfddd14f6',
 b'refs/pull/12/head': b'd2fcee067c4f003084950799f8cb408625d8c610',
 b'refs/pull/13/head': b'5d29d66bdadfb7e265b0f895ca1bf6f26f7bad39',
 b'refs/pull/14/head': b'114a4a2af63270ea69f08b5801cf1bdfa1c29222',
 b'refs/pull/15/head': b'ffe1f38e3a91ef69b784b867ad5d7729bf17499f',
 b'refs/pull/16/head': b'1b2ee6dbbe96435009ef96a80776ebc30e74b082',
 b'refs/pull/17/head': b'b1d52163d2170c1b7a8fe9a0d05bf15a7c4dfcb0',
 b'refs/pull/18/head': b'ac8a2c93c8c93985eb7a6e2742e2a3e5aa4786d8',
 b'refs/pull/19/head': b'2363a9a27b3223553f87ce2607a431b0b6ac9510',
 b'refs/pull/2/head': b'044c475cbccd47aad5dfbc3aa672ef37dc401fbc',
 b'refs/pull/20/head': b'163eadfc89d1e2a19ef87b979cd3f912b2606a31',
 b'refs/pull/21/head': b'd265b2309d471cb6edbcf349259837bd7401be10',
 b'refs/pull/22/head': b'4a7b4e9f3ab3b38abbf300c4c7ea216d804bf768',
 b'refs/pull/23/head': b'b3d3e249c98848e3f94cd266fcec31677cf51ba7',
 b'refs/pull/24/head': b'c6a5918b051d30688bf70f7113bc097d44f97353',
 b'refs/pull/25/head': b'cdc7a8a78d0344207a701370fe64452b8a32cbaa',
 b'refs/pull/26/head': b'cdc7a8a78d0344207a701370fe64452b8a32cbaa',
 b'refs/pull/27/head': b'40fee483349643993e1aa807b6bb92e70f8d0026',
 b'refs/pull/28/head': b'7fbbc778dbcf88f57ac70a1c8c47f51dcc7c044a',
 b'refs/pull/29/head': b'e3bd34e88a215a96d79dc362c3bda22966d1a159',
 b'refs/pull/3/head': b'f05bb4e34974fe56bde6329b65917eb6c13bb492',
 b'refs/pull/30/head': b'8f9f426f13d70b21f573f7c50bbe01e8ce38f158',
 b'refs/pull/31/head': b'8f9f426f13d70b21f573f7c50bbe01e8ce38f158',
 b'refs/pull/32/head': b'5e8e2a3d76050131a74e943045ce370cf05c8a5a',
 b'refs/pull/33/head': b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06',
 b'refs/pull/34/head': b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06',
 b'refs/pull/35/head': b'01394b26a91cbfdcdea7ceccf87b1d57dc8d954a',
 b'refs/pull/36/head': b'01394b26a91cbfdcdea7ceccf87b1d57dc8d954a',
 b'refs/pull/37/head': b'9329115eba6048db7972381ecb86bbc897d3f977',
 b'refs/pull/38/head': b'9329115eba6048db7972381ecb86bbc897d3f977',
 b'refs/pull/39/head': b'2922709aaef766e633d29ad29b559b3597ebce5f',
 b'refs/pull/4/head': b'4ced0ff539deaed8c0550d8b9375964cf4f301ea',
 b'refs/pull/40/head': b'2922709aaef766e633d29ad29b559b3597ebce5f',
 b'refs/pull/41/head': b'2922709aaef766e633d29ad29b559b3597ebce5f',
 b'refs/pull/42/head': b'a80543f6271e23effee307d9b1e43ae0346ed32c',
 b'refs/pull/43/head': b'a80543f6271e23effee307d9b1e43ae0346ed32c',
 b'refs/pull/44/head': b'2e6645a55e1a665b42880d10fa731909ca2cbc62',
 b'refs/pull/45/head': b'2e6645a55e1a665b42880d10fa731909ca2cbc62',
 b'refs/pull/46/head': b'2e6645a55e1a665b42880d10fa731909ca2cbc62',
 b'refs/pull/47/head': b'5701b39fa080dc72d44bb8ab3748272535a1f555',
 b'refs/pull/48/head': b'5701b39fa080dc72d44bb8ab3748272535a1f555',
 b'refs/pull/49/head': b'5701b39fa080dc72d44bb8ab3748272535a1f555',
 b'refs/pull/5/head': b'f1f7d2da22ebd871b40633480bbcf65c6e359183',
 b'refs/pull/50/head': b'abee9488b95fe174d8c70eaf81481ff75e6a0aeb',
 b'refs/pull/51/head': b'abee9488b95fe174d8c70eaf81481ff75e6a0aeb',
 b'refs/pull/52/head': b'abee9488b95fe174d8c70eaf81481ff75e6a0aeb',
 b'refs/pull/53/head': b'5c9060d676990842910378026a54f76159f14af1',
 b'refs/pull/54/head': b'5c9060d676990842910378026a54f76159f14af1',
 b'refs/pull/55/head': b'5c9060d676990842910378026a54f76159f14af1',
 b'refs/pull/56/head': b'4772475351cda80b6417c70faf574e2b129ae3ce',
 b'refs/pull/57/head': b'4772475351cda80b6417c70faf574e2b129ae3ce',
 b'refs/pull/58/head': b'4772475351cda80b6417c70faf574e2b129ae3ce',
 b'refs/pull/59/head': b'6c22d07d89bb18e3085ba95855c3e57f78b4efd9',
 b'refs/pull/6/head': b'09c70147a2697351b54ab4c6fa963674ebfaf762',
 b'refs/pull/60/head': b'6a35b22ac59b05e838bc6c2541f6a43fb0aad0f1',
 b'refs/pull/61/head': b'1ca56d99216ea8e12ee32e40131c54c85473531d',
 b'refs/pull/62/head': b'140948d0e0f9e93242643485667db698b91ca84b',
 b'refs/pull/63/head': b'e26ef132b990841aa0d5fcb4d1172519d7f47540',
 b'refs/pull/64/head': b'140948d0e0f9e93242643485667db698b91ca84b',
 b'refs/pull/65/head': b'feac3d41c796fce059cf555730ec274b8b850687',
 b'refs/pull/66/head': b'feac3d41c796fce059cf555730ec274b8b850687',
 b'refs/pull/67/head': b'feac3d41c796fce059cf555730ec274b8b850687',
 b'refs/pull/68/head': b'feac3d41c796fce059cf555730ec274b8b850687',
 b'refs/pull/69/head': b'7b952d878baa72b3f1a07b7ea5ee2836f7fdb4ba',
 b'refs/pull/7/head': b'204624e13a9e07770997f95bdf45af179f27e9c4',
 b'refs/pull/70/head': b'7b952d878baa72b3f1a07b7ea5ee2836f7fdb4ba',
 b'refs/pull/71/head': b'7b952d878baa72b3f1a07b7ea5ee2836f7fdb4ba',
 b'refs/pull/72/head': b'da4c00f7e53abceeb80efc9baae8d5a836d32f30',
 b'refs/pull/73/head': b'da4c00f7e53abceeb80efc9baae8d5a836d32f30',
 b'refs/pull/74/head': b'bd0515a6b4b6e6266013920694598ae113d48b2c',
 b'refs/pull/75/head': b'4372406c65f3e11d0138123d45aaa774c27fbe76',
 b'refs/pull/76/head': b'1f997c23eab00214e49b2def58b6a6e86bdf1cc2',
 b'refs/pull/77/head': b'4372406c65f3e11d0138123d45aaa774c27fbe76',
 b'refs/pull/78/head': b'bf346d6742f6558d1bc5c34cd71c63e4eadf22bd',
 b'refs/pull/79/head': b'6beaaa4db5fb1705f2468462bb40ff2c46341786',
 b'refs/pull/8/head': b'4546786737d692063b9c175b2e2b5035649e7aaa',
 b'refs/pull/80/head': b'4df4e6289e4644f21e400d8e148649e9881b4adc',
 b'refs/pull/81/head': b'2ba920d2d2c834f3fd9c570d2150a0f5dd9dc200',
 b'refs/pull/82/head': b'd3880bf360b7e36e8cd1ad2c0ee588ffd30611f4',
 b'refs/pull/83/head': b'f13f3bfd7e4ad3e7084b89b94d789e4e4a9c8e0b',
 b'refs/pull/84/head': b'7ddf22fdb933da4734fac522287c2c99105b8ced',
 b'refs/pull/85/head': b'4b7fd4e6cb22afc01020d2464be022ed4b055395',
 b'refs/pull/86/head': b'80fb1308d9d7576f2d04526f3ce72e197f9ae7de',
 b'refs/pull/87/head': b'11f32203327e26fd9bcd26c0d7af0c2ebe6266de',
 b'refs/pull/88/head': b'5961f169e5cd94319fc32ff3d4888469d4a9a293',
 b'refs/pull/89/head': b'4415bf6ee68993eca9ea6c81d978d5d9f541a8d4',
 b'refs/pull/9/head': b'24763e5554012f97af5e9926673bdf9cccdc79dc',
 b'refs/pull/90/head': b'4415bf6ee68993eca9ea6c81d978d5d9f541a8d4',
 b'refs/pull/91/head': b'cdd4da987285797630230d68d871395a3e59f563',
 b'refs/pull/92/head': b'c3f9f83ef09dfe6cf221302cc3cb5e601d419fe2',
 b'refs/pull/93/head': b'cdd4da987285797630230d68d871395a3e59f563',
 b'refs/pull/94/head': b'8df5667732c0040b0c5f472568e1c5ef2a514170',
 b'refs/pull/95/head': b'8df5667732c0040b0c5f472568e1c5ef2a514170',
 b'refs/pull/96/head': b'8ee75013f768869f5e7b15e48dc69937d49de1ec',
 b'refs/pull/97/head': b'8ee75013f768869f5e7b15e48dc69937d49de1ec',
 b'refs/pull/98/head': b'7d46e2e2574cbfb575f32206468e3d0c9ad04231',
 b'refs/pull/99/head': b'7d46e2e2574cbfb575f32206468e3d0c9ad04231'}
In [36]:
[key[1].decode() for key in config.keys() if key[0] == b"remote"]
Out[36]:
['origin', 'nima']

dulwich.porcelain.ls_remote

In [79]:
dulwich.porcelain.ls_remote(url)
Out[79]:
{b'HEAD': b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06',
 b'refs/heads/debian': b'618389f8300615ba7d31c4ba7b75fb94770391e1',
 b'refs/heads/dev': b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06',
 b'refs/heads/main': b'71f2b1d96cfa8596319686a5a98514ad4ac85506',
 b'refs/pull/1/head': b'b90bd029b4707a94640957cbfae73a631a9d83e0',
 b'refs/pull/1/merge': b'c767a6a929e599c245a4241477093554c1ab0d10',
 b'refs/pull/10/head': b'ea5b60d7fc34fce65ec4a503c20536d3e0d4587a',
 b'refs/pull/11/head': b'0d7e63476077d2ed51728823f3ce54b57a8287e6',
 b'refs/pull/12/head': b'd2fcee067c4f003084950799f8cb408625d8c610',
 b'refs/pull/13/head': b'5d29d66bdadfb7e265b0f895ca1bf6f26f7bad39',
 b'refs/pull/14/head': b'114a4a2af63270ea69f08b5801cf1bdfa1c29222',
 b'refs/pull/15/head': b'ffe1f38e3a91ef69b784b867ad5d7729bf17499f',
 b'refs/pull/16/head': b'1b2ee6dbbe96435009ef96a80776ebc30e74b082',
 b'refs/pull/17/head': b'b1d52163d2170c1b7a8fe9a0d05bf15a7c4dfcb0',
 b'refs/pull/18/head': b'ac8a2c93c8c93985eb7a6e2742e2a3e5aa4786d8',
 b'refs/pull/19/head': b'2363a9a27b3223553f87ce2607a431b0b6ac9510',
 b'refs/pull/2/head': b'044c475cbccd47aad5dfbc3aa672ef37dc401fbc',
 b'refs/pull/20/head': b'163eadfc89d1e2a19ef87b979cd3f912b2606a31',
 b'refs/pull/21/head': b'd265b2309d471cb6edbcf349259837bd7401be10',
 b'refs/pull/22/head': b'4a7b4e9f3ab3b38abbf300c4c7ea216d804bf768',
 b'refs/pull/23/head': b'b3d3e249c98848e3f94cd266fcec31677cf51ba7',
 b'refs/pull/24/head': b'c6a5918b051d30688bf70f7113bc097d44f97353',
 b'refs/pull/25/head': b'cdc7a8a78d0344207a701370fe64452b8a32cbaa',
 b'refs/pull/26/head': b'cdc7a8a78d0344207a701370fe64452b8a32cbaa',
 b'refs/pull/27/head': b'40fee483349643993e1aa807b6bb92e70f8d0026',
 b'refs/pull/28/head': b'7fbbc778dbcf88f57ac70a1c8c47f51dcc7c044a',
 b'refs/pull/29/head': b'e3bd34e88a215a96d79dc362c3bda22966d1a159',
 b'refs/pull/3/head': b'f05bb4e34974fe56bde6329b65917eb6c13bb492',
 b'refs/pull/30/head': b'8f9f426f13d70b21f573f7c50bbe01e8ce38f158',
 b'refs/pull/31/head': b'8f9f426f13d70b21f573f7c50bbe01e8ce38f158',
 b'refs/pull/32/head': b'5e8e2a3d76050131a74e943045ce370cf05c8a5a',
 b'refs/pull/33/head': b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06',
 b'refs/pull/34/head': b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06',
 b'refs/pull/4/head': b'4ced0ff539deaed8c0550d8b9375964cf4f301ea',
 b'refs/pull/5/head': b'f1f7d2da22ebd871b40633480bbcf65c6e359183',
 b'refs/pull/6/head': b'09c70147a2697351b54ab4c6fa963674ebfaf762',
 b'refs/pull/7/head': b'204624e13a9e07770997f95bdf45af179f27e9c4',
 b'refs/pull/8/head': b'4546786737d692063b9c175b2e2b5035649e7aaa',
 b'refs/pull/9/head': b'24763e5554012f97af5e9926673bdf9cccdc79dc'}

dulwich.objects.Commit

In [43]:
repo.head()
Out[43]:
b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06'
In [45]:
commit = repo[repo.head()]
commit
Out[45]:
<Commit b'2fd55f0a653bf8ec2e7ffda16c7b2d601167da06'>
In [46]:
type(commit)
Out[46]:
dulwich.objects.Commit
In [39]:
commit.message
Out[39]:
b'Update etc.sh'
In [40]:
dir(commit)
Out[40]:
['__bytes__',
 '__class__',
 '__cmp__',
 '__delattr__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__slots__',
 '__str__',
 '__subclasshook__',
 '_author',
 '_author_time',
 '_author_timezone',
 '_author_timezone_neg_utc',
 '_check_has_member',
 '_chunked_text',
 '_commit_time',
 '_commit_timezone',
 '_commit_timezone_neg_utc',
 '_committer',
 '_deserialize',
 '_encoding',
 '_extra',
 '_get_extra',
 '_get_parents',
 '_gpgsig',
 '_header',
 '_is_legacy_object',
 '_mergetag',
 '_message',
 '_needs_serialization',
 '_parents',
 '_parse_file',
 '_parse_legacy_object',
 '_parse_legacy_object_header',
 '_parse_object',
 '_parse_object_header',
 '_serialize',
 '_set_parents',
 '_sha',
 '_tree',
 'as_legacy_object',
 'as_legacy_object_chunks',
 'as_pretty_string',
 'as_raw_chunks',
 'as_raw_string',
 'author',
 'author_time',
 'author_timezone',
 'check',
 'commit_time',
 'commit_timezone',
 'committer',
 'copy',
 'encoding',
 'extra',
 'from_file',
 'from_path',
 'from_raw_chunks',
 'from_raw_string',
 'from_string',
 'get_type',
 'gpgsig',
 'id',
 'mergetag',
 'message',
 'parents',
 'raw_length',
 'set_raw_chunks',
 'set_raw_string',
 'set_type',
 'sha',
 'tree',
 'type',
 'type_name',
 'type_num']
In [41]:
main = heads.main
main
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_300/3703271362.py in <module>
----> 1 main = heads.main
      2 main

NameError: name 'heads' is not defined

Get the commit pointed to by head called master.

In [17]:
main.commit
Out[17]:
<git.Commit "95ed236bd715a06320ee85d519fb79a0adffe072">
In [18]:
main.rename("main2")
Out[18]:
<git.Head "refs/heads/main2">

Verify that the main branch has been renamed to main2.

In [19]:
!cd {dir_local} && git branch
* main2

Get the Active Branch

In [19]:
[m for m in dir(dulwich.porcelain) if "br" in m]
Out[19]:
['_make_branch_ref',
 '_update_head_during_checkout_branch',
 'active_branch',
 'branch_create',
 'branch_delete',
 'branch_list',
 'checkout_branch',
 'find_unique_abbrev',
 'get_branch_remote']
In [20]:
dulwich.porcelain.active_branch(repo)
Out[20]:
b'main'

Get All Branches

In [21]:
dulwich.porcelain.branch_list(repo)
Out[21]:
{b'main'}

Changed Files

Update a file.

In [23]:
!echo "# add a line of comment" >> {dir_local}/build.sh

Staged Files

The file build.sh is now staged.

Commit the change.

Push the Commits

Push the local main2 branch to the remote main2 branch.

The above is equivalent to the following more detailed specification.

Push the local main2 branch to the remote main branch.

git pull

In [11]:
!ls {dir_local}
abc  build.sh  readme.md
In [12]:
dulwich.porcelain.pull(repo, refspecs="main")
In [13]:
!ls {dir_local}
abc  build.sh  readme.md  test1.txt

git checkout

In [12]:
[m for m in dir(dulwich.porcelain) if 'checkout' in m]
Out[12]:
['_update_head_during_checkout_branch', 'checkout_branch']
In [18]:
!git -C {dir_local} status
On branch dev
nothing to commit, working tree clean
In [26]:
!git -C {dir_local} branch
* dev
  main
In [27]:
dulwich.porcelain.checkout_branch(repo, "main")
In [28]:
!git -C {dir_local} branch
  dev
* main

git tag

List all tags.

In [27]:
!git -C {dir_local} tag 
v1.0.0
In [24]:
[m for m in dir(dulwich.porcelain) if "tag" in m]
Out[24]:
['_make_tag_ref',
 'get_unstaged_changes',
 'print_tag',
 'show_tag',
 'tag_create',
 'tag_delete',
 'tag_list']
In [37]:
dulwich.porcelain.tag_list(repo)
Out[37]:
[b'v1.0.0', b'v2.0.0']

git tag tag_name

Create a new tag.

In [38]:
dulwich.porcelain.tag_create(repo, "v2.0.0")
In [40]:
dulwich.porcelain.push(repo, refspecs="v2.0.0")
---------------------------------------------------------------------------
HTTPUnauthorized                          Traceback (most recent call last)
Cell In[40], line 1
----> 1 dulwich.porcelain.push(repo, refspecs="v2.0.0")

File /usr/local/lib/python3.10/dist-packages/dulwich/porcelain.py:1181, in push(repo, remote_location, refspecs, outstream, errstream, force, **kwargs)
   1179 remote_location = client.get_url(path)
   1180 try:
-> 1181     result = client.send_pack(
   1182         path,
   1183         update_refs,
   1184         generate_pack_data=r.generate_pack_data,
   1185         progress=errstream.write,
   1186     )
   1187 except SendPackError as exc:
   1188     raise Error(
   1189         "Push to " + remote_location + " failed -> " + exc.args[0].decode(),
   1190     ) from exc

File /usr/local/lib/python3.10/dist-packages/dulwich/client.py:2015, in AbstractHttpGitClient.send_pack(self, path, update_refs, generate_pack_data, progress)
   1996 """Upload a pack to a remote repository.
   1997 
   1998 Args:
   (...)
   2012 
   2013 """
   2014 url = self._get_url(path)
-> 2015 old_refs, server_capabilities, url = self._discover_references(
   2016     b"git-receive-pack", url
   2017 )
   2018 (
   2019     negotiated_capabilities,
   2020     agent,
   2021 ) = self._negotiate_receive_pack_capabilities(server_capabilities)
   2022 negotiated_capabilities.add(capability_agent())

File /usr/local/lib/python3.10/dist-packages/dulwich/client.py:1940, in AbstractHttpGitClient._discover_references(self, service, base_url)
   1938     tail += "?service=%s" % service.decode("ascii")
   1939 url = urljoin(base_url, tail)
-> 1940 resp, read = self._http_request(url, headers)
   1942 if resp.redirect_location:
   1943     # Something changed (redirect!), so let's update the base URL
   1944     if not resp.redirect_location.endswith(tail):

File /usr/local/lib/python3.10/dist-packages/dulwich/client.py:2218, in Urllib3HttpGitClient._http_request(self, url, headers, data)
   2216     raise NotGitRepository()
   2217 if resp.status == 401:
-> 2218     raise HTTPUnauthorized(resp.headers.get("WWW-Authenticate"), url)
   2219 if resp.status == 407:
   2220     raise HTTPProxyUnauthorized(resp.headers.get("Proxy-Authenticate"), url)

HTTPUnauthorized: No valid credentials provided

git diff

In [16]:
help(repo.refs[4].commit.diff)
Help on method diff in module git.diff:

diff(other: Union[Type[git.diff.Diffable.Index], Type[ForwardRef('Tree')], object, NoneType, str] = <class 'git.diff.Diffable.Index'>, paths: Union[str, List[str], Tuple[str, ...], NoneType] = None, create_patch: bool = False, **kwargs: Any) -> 'DiffIndex' method of git.objects.commit.Commit instance
    Creates diffs between two items being trees, trees and index or an
    index and the working tree. It will detect renames automatically.
    
    :param other:
        Is the item to compare us with.
        If None, we will be compared to the working tree.
        If Treeish, it will be compared against the respective tree
        If Index ( type ), it will be compared against the index.
        If git.NULL_TREE, it will compare against the empty tree.
        It defaults to Index to assure the method will not by-default fail
        on bare repositories.
    
    :param paths:
        is a list of paths or a single path to limit the diff to.
        It will only include at least one of the given path or paths.
    
    :param create_patch:
        If True, the returned Diff contains a detailed patch that if applied
        makes the self to other. Patches are somewhat costly as blobs have to be read
        and diffed.
    
    :param kwargs:
        Additional arguments passed to git-diff, such as
        R=True to swap both sides of the diff.
    
    :return: git.DiffIndex
    
    :note:
        On a bare repository, 'other' needs to be provided as Index or as
        as Tree/Commit, or a git command error will occur

In [3]:
url = "https://github.com/dclong/docker-ubuntu_b.git"
dir_local = "/tmp/" + url[(url.rindex("/") + 1) :]
!rm -rf {dir_local}
In [4]:
repo = git.Repo.clone_from(url, dir_local, branch="main")
repo
Out[4]:
<git.repo.base.Repo '/tmp/docker-ubuntu_b.git/.git'>
In [25]:
repo.refs
Out[25]:
[<git.Head "refs/heads/debian">,
 <git.Head "refs/heads/dev">,
 <git.Head "refs/heads/main">,
 <git.RemoteReference "refs/remotes/origin/HEAD">,
 <git.RemoteReference "refs/remotes/origin/debian">,
 <git.RemoteReference "refs/remotes/origin/dev">,
 <git.RemoteReference "refs/remotes/origin/main">]
In [6]:
diffs = repo.refs[4].commit.diff(repo.refs[3].commit)
diffs
Out[6]:
[]
In [21]:
diffs = repo.refs[4].commit.diff(repo.refs[2].commit)
diffs
Out[21]:
[<git.diff.Diff at 0x7f0eb05d1a60>, <git.diff.Diff at 0x7f0eb05d1af0>]
In [13]:
str(diffs[0])
Out[13]:
'Dockerfile\n=======================================================\nlhs: 100644 | 8ae5c7650a8c031a8e176d896a3665bbe7e2aae8\nrhs: 100644 | 9f2304d9a97aa1279ad1938b3bb74790172c9d8b'
In [12]:
repo.refs[5].name
Out[12]:
'origin/main'
In [6]:
print(repo.git.status())
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
In [6]:
repo.git.checkout("debian", force=True)
Out[6]:
"Your branch is up to date with 'origin/debian'."
In [8]:
repo.git.checkout(b="a_new_branch", force=True)
Out[8]:
''
In [ ]:
nima = repo.refs[4].checkout(force=True, b="nima")
nima
In [50]:
diffs = nima.commit.diff(repo.refs[-1].commit)
diffs[0].diff
Out[50]:
''

Diff the dev and the main branch, which is equivalent to the Git command git diff dev..main.

In [30]:
repo.refs[2].commit.diff(repo.refs[1].commit)
Out[30]:
[]
In [32]:
diffs = repo.refs[2].commit.diff(repo.refs[0].commit)
diffs
Out[32]:
[<git.diff.Diff at 0x128ca3a60>]
In [33]:
diffs[0]
Out[33]:
<git.diff.Diff at 0x128ca3a60>
In [24]:
diffs = repo.refs[6].commit.diff(repo.refs[7].commit)
diffs
Out[24]:
[]
In [25]:
diffs = repo.refs[4].commit.diff(repo.refs[7].commit)
diffs
Out[25]:
[<git.diff.Diff at 0x128ca3790>]
In [26]:
diffs[0].diff
Out[26]:
''
In [27]:
diffs = repo.refs[7].commit.diff(repo.refs[4].commit)
diffs
Out[27]:
[<git.diff.Diff at 0x128ca3820>]
In [28]:
diffs[0].diff
Out[28]:
''
In [19]:
any(ele for ele in [""])
Out[19]:
False
In [23]:
repo.branches[0].name
Out[23]:
'dev'
In [ ]:
for branch in repo.branches:
    branch.
In [12]:
commit = repo.head.commit
commit
Out[12]:
<git.Commit "6716bb0d016bd63ba543f3d9c67a65dadecd152e">
In [15]:
type(repo.branches[0])
Out[15]:
git.refs.head.Head
In [17]:
repo.refs[4].commit.diff(repo.refs[2].commit)
Out[17]:
[]
In [9]:
repo.refs[4].commit.diff(repo.refs[3].commit)
Out[9]:
[<git.diff.Diff at 0x127370310>]
In [20]:
help(repo.git.branch)
Help on function <lambda> in module git.cmd:

<lambda> lambda *args, **kwargs

In [28]:
repo.heads
Out[28]:
[<git.Head "refs/heads/dev">, <git.Head "refs/heads/main">]

Diff the debian and the main branches but limit diff to specified paths (via the paths parameter).

In [24]:
diffs = repo.refs[4].commit.diff(repo.refs[2].commit, paths=["build.sh", "scripts"])
diffs
Out[24]:
[]

Comments