Get some help with Bitbucket hell

Introduction

Over the last couple of years, I’ve seen an upward trend in the number of internal git repositories organisations are managing. In the past organisations might create a handful of repositories aligned with each architectural layer, i.e. front end, backend, analytics and infrastructure. Today it’s not uncommon for a small organisation to be managing hundreds of micro repositories.

This trend seems to be driven by a couple of factors.

The rise of microservices has brought with it a whole set of new architectural principles and opinions. Microservices architectures seek to achieve a single codebase and deployment per service. The goal is to ensure sharded nothing or isolation between services. A single base per service gets interpreted as a single repository per service.

The second argument for small repositories is it reduces build time on feature branches. The simplest way to trigger a build is when we see a new commit. Simple pipelines with large repos needlessly rebuild modules, which haven’t changed.

The final reason is it merely takes more upfront thinking and effort to create a taxonomy to group related modules. Creating a new repository costs almost nothing.

Given large numbers of repositories are our new reality, I wanted a command-line tool to run commands across repos.

Bitbucket-helper is a command-line utility for working with lots of bitbucket server projects and repositories. This handy helper allows you to clone or pull every bitbucket repository you have access too. Repositories are pathed under their parent project, making it easy to identify the source project.

Installation

The utility requires python3.7.

pip install bitbucket-helper

Setup

The first time you attempt to sync, you will get prompted to provide your bitbucket details. The tool supports both Bitbucket Server and Bitbucket Cloud.

Bitbucket Server requires a read-only personal access token. To generate goto Bitbucket -> Manage account -> Personal Access Tokens.

BitBucket Cloud requires an App Password if you use 2FA. The App needs to be granted read access.

All configuration settings get stored in ~/.bitbucket-helper.config. To reconfigure you can delete this file.

Features

Listing Repositories

To list all of the repositories, you have permission to access:

bitbucket-helper repo

The command outputs the following quoted comma-separated values:

"project_key","clone_uri"

Synchronising

The synchronisation function uses bitbucket APIs to get all accessible projects and repositories. For each project, it will create a directory using its key. For each repository, we perform a git clone or a git pull if we already have a local copy. After the pull, we prune and delete local branches that have been merged at the origin.

Local directory structure:

• working_directory (~/bitbucket-server)
  • proj1
    • repo1
    • repo2
  • proj2
    • repo1
    • repo2

bitbucket-helper sync

Pull requests

Bitbucket helper can list pull requests across all of your repositories. By default it returns OPEN pull requests, you can also filter for pull requests in a specific state, e.g. ALL, OPEN, MERGED, DECLINED.

bitbucket-helper pr --state=MERGED

Git log commands

Bitbucket helper makes it easy to search for commits across all of your repositories using simple filters. Log commands operate on locally synched repositories, so remember to do a bitbucket-helper sync first.

You can find all commits after a specified date using:

bitbucket-helper log --after=2020-03-02

To search from commits between two tags:

bitbucket-helper log --from_tag=1.9.0 --to_tag=1.9.1

Resources

• PiPy
• GitHub

• 2022 1
• 2021 3
• 2020 4
• 2016 2
• 2013 1