fileserver/lib/Botan-3.2.0/doc/dev_ref/contributing.rst

Notes for New Contributors
===================================

Source Code Layout
-------------------------------------------------

Under ``src`` there are directories

* ``lib`` is the library itself, more on that below
* ``cli`` is the command line application ``botan``
* ``tests`` contain what you would expect. Input files go under ``tests/data``.
* ``python/botan3.py`` is the Python ctypes wrapper
* ``bogo_shim`` contains the shim binary and configuration for
  `BoringSSL's TLS test suite <https://github.com/google/boringssl/tree/master/ssl/test>`_
* ``fuzzer`` contains fuzz targets for various modules of the library
* ``build-data`` contains files read by the configure script. For
  example ``build-data/cc/gcc.txt`` describes various gcc options.
* ``examples`` contains usage examples used in the documentation.
* ``scripts`` contains misc scripts: install, distribution, various
  codegen things. Scripts controlling CI go under ``scripts/ci``.
* ``configs`` contains configuration files tools like pylint
* ``editors`` contains configuration files for editors like vscode and emacs

Under ``doc`` one finds the sources of this documentation

Library Layout
----------------------------------------

Under ``src/lib`` are several directories

* ``asn1`` is the DER encoder/decoder
* ``base`` defines some high level types
* ``block`` contains the block cipher implementations
* ``codec`` has hex, base64, base32, base58
* ``compat`` a (partial) compatibility layer for the libsodium API
* ``compression`` has the compression wrappers (zlib, bzip2, lzma)
* ``entropy`` has various entropy sources used by some of the RNGs
* ``ffi`` is the C99 API
* ``filters`` is a filter/pipe API for data transforms
* ``hash`` contains the hash function implementations
* ``kdf`` contains the key derivation functions
* ``mac`` contains the message authentication codes
* ``math`` is the big integer math library. It is divided into three parts:
  ``mp`` which are the low level algorithms; ``bigint`` which is a C++ wrapper
  around ``mp``, and ``numbertheory`` which contains higher level algorithms like
  primality testing and exponentiation
* ``misc`` contains odds and ends: format preserving encryption, SRP, threshold
  secret sharing, all or nothing transform, and others
* ``modes`` contains block cipher modes (CBC, GCM, etc)
* ``passhash`` contains password hashing algorithms for authentication
* ``pbkdf`` contains password hashing algorithms for key derivation
* ``pk_pad`` contains padding schemes for public key algorithms
* ``prov`` contains bindings to external libraries such as PKCS #11
* ``psk_db`` contains a generic interface for a Pre-Shared-Key database
* ``pubkey`` contains the public key algorithms
* ``rng`` contains the random number generators
* ``stream`` contains the stream ciphers
* ``tls`` contains the TLS implementation
* ``utils`` contains various utility functions and types
* ``x509`` is X.509 certificates, PKCS #10 requests, OCSP

Each of these folders can contain subfolders which are treated as modules if they
contain an ``info.txt`` file. These submodules have an implicit dependency on their
parent module. The chapter :ref:`configure_script` contains more information on
Botan's module architecture.

Sending patches
----------------------------------------

All contributions should be submitted as pull requests via GitHub
(https://github.com/randombit/botan). If you are planning a large
change, open a discussion ticket on github before starting out to make
sure you are on the right path. And once you have something written,
even if it is not complete/ready to go, feel free to open a draft PR
for early review and comment.

If possible please sign your git commits using a PGP key.
See https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work for
instructions on how to set this up.

Depending on what your change is, your PR should probably also include an update
to ``news.rst`` with a note explaining the change. If your change is a
simple bug fix, a one sentence description is perhaps sufficient. If there is an
existing ticket on GitHub with discussion or other information, reference it in
your change note as 'GH #000'.

Update ``doc/credits.txt`` with your information so people know what you did!

If you are interested in contributing but don't know where to start check out
``doc/dev_ref/todo.rst`` for some ideas - these are changes we would almost
certainly accept once they've passed code review.

Also, try building and testing it on whatever hardware you have handy,
especially unusual platforms, or using C++ compilers other than the regularly
tested GCC, Clang, and Visual Studio.

FFI Additions
----------------

If adding a new function declaration to ``ffi.h``, the same PR must also add the
same declaration in the Python binding ``botan3.py``, in addition the new API
functionality must be exposed to Python and a test written in Python.

Git Usage
----------------------------------------

Do *NOT* merge ``master`` into your topic branch, this creates needless commits
and noise in history. Instead, as needed, rebase your branch against master
(``git rebase -i master``) and force push the branch to update the PR. If the
GitHub PR page does not report any merge conflicts and nobody asks you to
rebase, you don't need to rebase.

Try to keep your history clean and use rebase to squash your commits as
needed. If your diff is less than roughly 100 lines, it should probably be a
single commit. Only split commits as needed to help with review/understanding of
the change.

Python
----------------------------------------

Scripts should be in Python 3 whenever possible.

For configure.py (and helper scripts install.py, cleanup.py and build_docs.py)
the target is stock (no modules outside the standard library) CPython 3.x.
Support for PyPy, etc is great when viable (in the sense of not causing problems
for 3.x, and not requiring huge blocks of version dependent code). As running
this program successfully is required for a working build, making it as portable
as possible is considered key.

The python wrapper botan3.py targets CPython 3.x, and latest PyPy. Note that
a single file is used to avoid dealing with any of Python's various crazy module
distribution issues.

For random scripts not typically run by an end-user (codegen, visualization, and
so on) there isn't any need to worry about platform independence. Here it's fine
to depend on any useful modules such as graphviz or matplotlib, regardless if it
is available from a stock CPython install.

Build Tools and Hints
----------------------------------------

If you don't already use it for all your C/C++ development, install ``ccache``
(or on Windows, ``sccache``) right now, and configure a large cache on a fast
disk. It allows for very quick rebuilds by caching the compiler output.

Use ``--enable-sanitizers=`` flag to enable various sanitizer checks.  Supported
values including "address" and "undefined" for GCC and Clang. GCC also supports
"iterator" (checked iterators), and Clang supports "memory" (MSan) and
"coverage" (for fuzzing).

On Linux if you have the ``lcov`` and ``gcov`` tools installed, then running
``./src/scripts/ci_build.py coverage`` will produce a coverage enabled build,
run the tests, test the fuzzers against a corpus, and produce an HTML report
of total coverage. This coverage build requires the development headers for
zlib, bzip2, liblzma, TrouSerS (libtspi), and Sqlite3.

Copyright Notice
----------------------------------------

At the top of any new file add a comment with a copyright and a reference to the
license, for example::

  /*
  * (C) 202x <You>
  *
  * Botan is released under the Simplified BSD License (see license.txt)
  */

If you are making a substantial or non-trivial change to an existing file, add
or update your own copyright statement at the top of each file.

Style Conventions
----------------------------------------

When writing your code remember the need for it to be easily understood by
reviewers and auditors, both at the time of the patch submission and in the
future.

Avoid complicated template metaprogramming where possible. It has its places but
should be used judiciously.

When designing a new API (for use either by library users or just internally)
try writing out the calling code first. That is, write out some code calling
your idealized API, then just implement that API.  This can often help avoid
cut-and-paste by creating the correct abstractions needed to solve the problem
at hand.

The C++11 ``auto`` keyword is very convenient but only use it when the type
truly is obvious (considering also the potential for unexpected integer
conversions and the like, such as an apparent uint8_t being promoted to an int).

Unless there is a specific reason otherwise (eg due to calling some C API which
requires exactly a ``long*`` be provided) integer types should be either
``(u)intXX_t`` or ``size_t``. If the variable is used for integer values of "no
particular size", as in the loop ``for(some_type i = 0; i != 100; ++i)`` then
the type should be ``size_t``. Use one of the specific size integer types only
when there is a algorithmic/protocol reason to use an integer of that size. For
example if a parsing a protocol that uses 16-bit integer fields to encode a
length, naturally one would use ``uint16_t`` there.

If a variable is defined and not modified, declare it ``const``.  Some exception
for very short-lived variables, but generally speaking being able to read the
declaration and know it will not be modified is useful.

Use ``override`` annotations whenever overriding a virtual function.  If
introducing a new type that is not intended for further derivation, mark it ``final``.

Avoid explicit ``new`` or (especially) explicit ``delete``: use RAII,
``make_unique``, etc.

Use ``m_`` prefix on all member variables.

``clang-format`` is used for all C++ formatting. The configuration is
in ``.clang-format`` in the root directory. You can rerun the
formatter using ``make fmt`` or by invoking the script
``src/scripts/dev_tools/run_clang_format.py``. If the output would be
truly horrible, it is allowed to disable formatting for a specific
area using ``// clang-format off`` annotations.

.. note::

   Since the output of clang-format varies from version to version, we
   currently require using exactly ``clang-format 15``.

Use braces on both sides of if/else blocks, even if only using a single
statement.

Avoid ``using namespace`` declarations, even inside of single functions.  One
allowed exception is ``using namespace std::placeholders`` in functions which
use ``std::bind``. (But, don't use ``std::bind`` - use a lambda instead).

Use ``::`` to explicitly refer to the global namespace (eg, when calling an OS
or external library function like ``::select`` or ``::sqlite3_open``).

Use of External Dependencies
----------------------------------------

Compiler Dependencies
~~~~~~~~~~~~~~~~~~~~~~~

The library should always be as functional as possible when compiled with just
Standard C++20. However, feel free to use the full language.

Use of compiler extensions is fine whenever appropriate; this is typically
restricted to a single file or an internal header. Compiler extensions used
currently include native uint128_t, SIMD intrinsics, inline asm syntax and so
on, so there are some existing examples of appropriate use.

Generally intrinsics or inline asm is preferred over bare assembly to avoid
calling convention issues among different platforms; the improvement in
maintainability is seen as worth any potential performance tradeoff. One risk
with intrinsics is that the compiler might rewrite your clever const-time SIMD
into something with a conditional jump, but code intended to be const-time
should in any case be annotated (using ``CT::poison``) so it can be checked at
runtime with tools.

Operating System Dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you're adding a small OS dependency in some larger piece of code, try to
contain the actual non-portable operations to utils/os_utils.* and then call
them from there.

As a policy, operating systems which are not supported by their original vendor
are not supported by Botan either. Patches that complicate the code in order to
support obsolete operating systems will likely be rejected. In writing OS
specific code, feel free to assume roughly POSIX 2008, or for Windows, Windows 8
/Server 2012 (which are as of this writing the oldest versions still supported
by Microsoft).

Some operating systems, such as OpenBSD, only support the latest release. For
such cases, it's acceptable to add code that requires APIs added in the most
recent release of that OS as soon as the release is available.

Library Dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Any external library dependency - even optional ones - is met with as one PR
submitter put it "great skepticism".

At every API boundary there is potential for confusion that does not exist when
the call stack is all contained within the boundary.  So the additional API
really needs to pull its weight. For example a simple text parser or such which
can be trivially implemented is not really for consideration. As a rough idea of
the bar, equate the viewed cost of an external dependency as at least 1000
additional lines of code in the library. That is, if the library really does
need this functionality, and it can be done in the library for less than that,
then it makes sense to just write the code. Yup.

Currently the (optional) external dependencies of the library are several
compression libraries (zlib, bzip2, lzma), sqlite3 database, Trousers (TPM
integration), plus various operating system utilities like basic filesystem
operations. These provide major pieces of functionality which seem worth the
trouble of maintaining an integration with.

At this point the most plausible examples of an appropriate new external
dependency are all deeper integrations with system level cryptographic
interfaces (CommonCrypto, CryptoAPI, /dev/crypto, iOS keychain, TPM 2.0, etc)