root/fileserver
1
0
Fork 0
Code Issues 17 Actions Wiki Activity

Rewrote the server in cpp with the frontend in svelte

Browse Source
This commit is contained in:
Mutzi
2023-10-20 13:02:21 +02:00
commit 03b22ebb61
4168 changed files with 831370 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

440
lib/Botan-3.2.0/doc/dev_ref/configure.rst Normal file
View File

@@ -0,0 +1,440 @@
.. _configure_script:
Understanding configure.py
============================
.. highlight:: none
Botan's build is handled with a custom Python script, ``configure.py``.
This document tries to explain how configure works.
.. note::
You only need to read this if you are modifying the library,
or debugging some problem with your build. For how to use it,
see :ref:`building`.
Build Structure
--------------------
Modules are a group of related source and header files, which can be
individually enabled or disabled at build time. Modules can depend on
other modules; if a dependency is not available then the module itself
is also removed from the list. Examples of modules in the existing
codebase are ``asn1`` and ``x509``, Since ``x509`` depends on (among
other things) ``asn1``, disabling ``asn1`` will also disable ``x509``.
Most modules define one or more macros, which application code can use
to detect the modules presence or absence. The value of each macro is
a datestamp, in the form YYYYMMDD which indicates the last time this
module changed in a way that would be visible to an application. For
example if a class gains a new function, the datestamp should be
incremented. That allows applications to detect if the new feature is
available.
What ``configure.py`` does
-----------------------------
First, all command line options are parsed.
Then all of the files giving information about target CPUs, compilers,
etc are parsed and sanity checked.
In ``calculate_cc_min_version`` the compiler version is detected using
the preprocessor.
Then in ``check_compiler_arch`` the target architecture are detected, again
using the preprocessor.
Now that the target is identified and options have been parsed, the modules to
include into the artifact are picked, in ``ModulesChooser``.
In ``create_template_vars``, a dictionary of variables is created which describe
different aspects of the build. These are serialized to
``build/build_config.json``.
Up until this point no changes have been made on disk. This occurs in
``do_io_for_build``. Build output directories are created, and header files are
linked into ``build/include/botan``. Templates are processed to create the
Makefile, ``build.h`` and other artifacts.
When Modifying ``configure.py``
--------------------------------
Run ``./src/scripts/ci_build.py lint`` to run Pylint checks after any change.
Template Language
--------------------
Various output files are generated by processing input files using a simple
template language. All input files are stored in ``src/build-data`` and use the
suffix ``.in``. Anything not recognized as a template command is passed through
to the output unmodified. The template elements are:
* Variable substitution, ``%{variable_name}``. The configure script creates
many variables for various purposes, this allows getting their value within
the output. If a variable is not defined, an error occurs.
If a variable reference ends with ``|upper``, the value is uppercased before
being inserted into the template output.
* Iteration, ``%{for variable} block %{endfor}``. This iterates over a list and
repeats the block as many times as it is included. Variables within the block
are expanded. The two template elements ``%{for ...}`` and ``%{endfor}`` must
appear on lines with no text before or after.
* Conditional inclusion, ``%{if variable} block %{endif}``. If the variable
named is defined and true (in the Python sense of the word; if the variable
is empty or zero it is considered false), then the block will be included and
any variables expanded. As with the for loop syntax, both the start and end
of the conditional must be on their own lines with no additional text.
Build.h
-------
The ``build.h`` header file is generated and overwritten each time the
``configure.py`` script is executed. This header can be included in any header
or source file and provides plenty of compile-time information in the form of
preprocessor ``#define``\ s.
It is helpful to check which modules are included in the current build of the
library via macro defines of the form "BOTAN_HAS" followed by the module name.
Also, it contains :ref:`version information macros <versioning>` and compile-time
library configurations.
Adding a new module
--------------------
Create a directory in the appropriate place and create a ``info.txt`` file.
Syntax of ``info.txt``
------------------------
.. warning::
The syntax described here is documented to make it easier to use
and understand, but it is not considered part of the public API
contract. That is, the developers are allowed to change the syntax
at any time on the assumption that all users are contained within
the library itself. If that happens this document will be updated.
Modules and files describing information about the system use the same
parser and have common syntactical elements.
Comments begin with '#' and continue to end of line.
There are three main types: maps, lists, and variables.
A map has a syntax like::
<MAP_NAME>
NAME1 -> VALUE1
NAME2 -> VALUE2
...
</MAP_NAME>
The interpretation of the names and values will depend on the map's name
and what type of file is being parsed.
A list has similar syntax, it just doesn't have values::
<LIST_NAME>
ELEM1
ELEM2
...
</LIST_NAME>
Lastly there are single value variables like::
VAR1 SomeValue
VAR2 "Quotes Can Be Used (And will be stripped out)"
VAR3 42
Variables can have string, integer or boolean values. Boolean values
are specified with 'yes' or 'no'.
Module Syntax
---------------------
The ``info.txt`` files have the following elements. Not all are required; a minimal
file for a module with no dependencies might just contain a macro define and module_info.
Lists:
* ``comment`` and ``warning`` provides block-comments which
are displayed to the user at build time.
* ``requires`` is a list of module dependencies. An ``os_features`` can be
specified as a condition for needing the dependency by writing it before
the module name and separated by a ``?``, e.g. ``rtlgenrandom?dyn_load``.
* ``header:internal`` is the list of headers (from the current module)
which are internal-only.
* ``header:public`` is a the list of headers (from the
current module) which should be exported for public use. If neither
``header:internal`` nor ``header:public`` are used then all headers
in the current directory are assumed internal.
.. note:: If you omit a header from both internal and public lists, it will
be ignored.
* ``header:external`` is used when naming headers which are included
in the source tree but might be replaced by an external version. This is used
for the PKCS11 headers.
* ``arch`` is a list of architectures this module may be used on.
* ``isa`` lists ISA features which must be enabled to use this module.
Can be proceeded by an ``arch`` name followed by a ``:`` if it is only needed
on a specific architecture, e.g. ``x86_64:ssse3``.
* ``cc`` is a list of compilers which can be used with this module. If the
compiler name is suffixed with a version (like "gcc:5.0") then only compilers
with that minimum version can use the module. If you need to exclude just
one specific compiler (for example because that compiler miscompiles the code
in the module), you can prefix a compiler name with ``!`` - like ``!msvc``.
* ``os_features`` is a list of OS features which are required in order to use this
module. Each line can specify one or more features combined with ','. Alternatives
can be specified on additional lines.
Maps:
* ``defines`` is a map from macros to datestamps. These macros will be defined in
the generated ``build.h``.
* ``module_info`` contains documentation-friendly information about the module.
Available mappings:
* ``name`` must contain a human-understandable name for the module
* ``brief`` may provide a short description about the module's contents
* ``type`` specifies the type of the module (defaults to ``Public``)
* ``Public`` Library users can directly interact with this module. E.g.
they may enable or disable the module at will during build.
* ``Internal`` Library users must not directly interact with this module.
It is enabled and used as required by other modules.
* ``Virtual`` This module does not contain any implementation but acts as
a container for other sub-modules. It cannot be interacted with by the
library user and cannot be depended upon directly.
* ``libs`` specifies additional libraries which should be linked if this module is
included. It maps from the OS name to a list of libraries (comma seperated).
* ``frameworks`` is a macOS/iOS specific feature which maps from an OS name to
a framework.
Variables:
* ``load_on`` Can take on values ``never``, ``always``, ``auto``, ``dep`` or ``vendor``.
TODO describe the behavior of these
* ``endian`` Required endian for the module (``any`` (default), ``little``, ``big``)
An example::
# Disable this by default
load_on never
<isa>
sse2
</isa>
<defines>
DEFINE1 -> 20180104
DEFINE2 -> 20190301
</defines>
<module_info>
name -> "This Is Just To Say"
brief -> "Contains a poem by William Carlos Williams"
</module_info>
<comment>
I have eaten
the plums
that were in
the icebox
</comment>
<warning>
There are no more plums
</warning>
<header:public>
header1.h
</header:public>
<header:internal>
header_helper.h
whatever.h
</header:internal>
<arch>
x86_64
</arch>
<cc>
gcc:4.9 # gcc 4.8 doesn't work for <reasons>
clang
</cc>
# Can work with POSIX+getentropy or Win32
<os_features>
posix1,getentropy
win32
</os_features>
<frameworks>
macos -> FramyMcFramerson
</frameworks>
<libs>
qnx -> foo,bar,baz
solaris -> socket
</libs>
Supporting a new CPU type
---------------------------
CPU information is stored in ``src/build-data/arch``.
There is also a file ``src/build-data/detect_arch.cpp`` which is used
for build-time architecture detection using the compiler preprocessor.
Supporting this is optional but recommended.
Lists:
* ``aliases`` is a list of alternative names for the CPU architecture.
* ``isa_extensions`` is a list of possible ISA extensions that can be used on
this architecture. For example x86-64 has extensions "sse2", "ssse3",
"avx2", "aesni", ...
Variables:
* ``endian`` if defined should be "little" or "big". This can also be
controlled or overridden at build time.
* ``family`` can specify a family group for several related architecture.
For example both x86_32 and x86_64 use ``family`` of "x86".
* ``wordsize`` is the default wordsize, which controls the size of limbs
in the multi precision integers. If not set, defaults to 32.
Supporting a new compiler
---------------------------
Compiler information is stored in ``src/build-data/cc``. Looking over
those files will probably help understanding, especially the ones for
GCC and Clang which are most complete.
In addition to the info file, for compilers there is a file
``src/build-data/detect_version.cpp``. The ``configure.py`` script runs the
preprocessor over this file to attempt to detect the compiler
version. Supporting this is not strictly necessary.
Maps:
* ``binary_link_commands`` gives the command to use to run the linker,
it maps from operating system name to the command to use. It uses
the entry "default" for any OS not otherwise listed.
* ``cpu_flags_no_debug`` unused, will be removed
* ``cpu_flags`` used to emit CPU specific flags, for example LLVM
bitcode target uses ``-emit-llvm`` flag. Rarely needed.
* ``isa_flags`` maps from CPU extensions (like NEON or AES-NI) to
compiler flags which enable that extension. These have the same name
as the ISA flags listed in the architecture files.
* ``lib_flags`` has a single possible entry "debug" which if set maps
to additional flags to pass when building a debug library.
Rarely needed.
* ``mach_abi_linking`` specifies flags to enable when building and
linking on a particular CPU. This is usually flags that modify
ABI. There is a special syntax supported here
"all!os1,arch1,os2,arch2" which allows setting ABI flags which are
used for all but the named operating systems and/or architectures.
* ``sanitizers`` is a map of sanitizers the compiler supports. It must
include "default" which is a list of sanitizers to include by default
when sanitizers are requested. The other keys should map to compiler
flags.
* ``so_link_commands`` maps from operating system to the command to
use to build a shared object.
Variables:
* ``binary_name`` the default name of the compiler binary.
* ``linker_name`` the name of the linker to use with this compiler.
* ``macro_name`` a macro of the for ``BOTAN_BUILD_COMPILER_IS_XXX``
will be defined.
* ``output_to_object`` (default "-o") gives the compiler option used to
name the output object.
* ``output_to_exe`` (default "-o") gives the compiler option used to
name the output object.
* ``add_include_dir_option`` (default "-I") gives the compiler option used
to specify an additional include dir.
* ``add_lib_dir_option`` (default "-L") gives the compiler option used
to specify an additional library dir.
* ``add_sysroot_option`` gives the compiler option used to specify the sysroot.
* ``add_lib_option`` (default "-l%s") gives the compiler option to
link in a library. ``%s`` will be replaced with the library name.
* ``add_framework_option`` (default "-framework") gives the compiler option
to add a macOS framework.
* ``preproc_flags`` (default "-E") gives the compiler option used to run
the preprocessor.
* ``compile_flags`` (default "-c") gives the compiler option used to compile a file.
* ``debug_info_flags`` (default "-g") gives the compiler option used to enable debug info.
* ``optimization_flags`` gives the compiler optimization flags to use.
* ``size_optimization_flags`` gives compiler optimization flags to use when
compiling for size. If not set then ``--optimize-for-size`` will use
the default optimization flags.
* ``sanitizer_optimization_flags`` gives compiler optimization flags to use
when building with sanitizers.
* ``coverage_flags`` gives the compiler flags to use when generating coverage
information.
* ``stack_protector_flags`` gives compiler flags to enable stack overflow checking.
* ``shared_flags`` gives compiler flags to use when generation shared libraries.
* ``lang_flags`` gives compiler flags used to enable the required version of C++.
* ``lang_binary_linker_flags`` gives flags to be passed to the linker when creating a binary
* ``warning_flags`` gives warning flags to enable.
* ``maintainer_warning_flags`` gives extra warning flags to enable during maintainer
mode builds.
* ``visibility_build_flags`` gives compiler flags to control symbol visibility
when generation shared libraries.
* ``visibility_attribute`` gives the attribute to use in the ``BOTAN_DLL`` macro
to specify visibility when generation shared libraries.
* ``ar_command`` gives the command to build static libraries
* ``ar_options`` gives the options to pass to ``ar_command``, if not set here
takes this from the OS specific information.
* ``ar_output_to`` gives the flag to pass to ``ar_command`` to specify where to
output the static library.
* ``werror_flags`` gives the complier flags to treat warnings as errors.
Supporting a new OS
---------------------------
Operating system information is stored in ``src/build-data/os``.
Lists:
* ``aliases`` is a list of alternative names which will be accepted
* ``target_features`` is a list of target specific OS features. Some of these
are supported by many OSes (for example "posix1") others are specific to
just one or two OSes (such as "getauxval"). Adding a value here causes a new
macro ``BOTAN_TARGET_OS_HAS_XXX`` to be defined at build time. Use
``configure.py --list-os-features`` to list the currently defined OS
features.
* ``feature_macros`` is a list of macros to define.
Variables:
* ``ar_command`` gives the command to build static libraries
* ``ar_options`` gives the options to pass to ``ar_command``
* ``ar_output_to`` gives the flag to pass to ``ar_command`` to specify where to
output the static library.
* ``bin_dir`` (default "bin") specifies where binaries should be installed,
relative to install_root.
* ``cli_exe_name`` (default "botan") specifies the name of the command line utility.
* ``default_compiler`` specifies the default compiler to use for this OS.
* ``doc_dir`` (default "doc") specifies where documentation should be installed,
relative to install_root
* ``header_dir`` (default "include") specifies where include files
should be installed, relative to install_root
* ``install_root`` (default "/usr/local") specifies where to install
by default.
* ``lib_dir`` (default "lib") specifies where library should be installed,
relative to install_root.
* ``lib_prefix`` (default "lib") prefix to add to the library name
* ``library_name``
* ``man_dir`` specifies where man files should be installed, relative to install_root
* ``obj_suffix`` (default "o") specifies the suffix used for object files
* ``program_suffix`` (default "") specifies the suffix used for executables
* ``shared_lib_symlinks`` (default "yes) specifies if symbolic names should be
created from the base and patch soname to the library name.
* ``soname_pattern_abi``
* ``soname_pattern_base``
* ``soname_pattern_patch``
* ``soname_suffix`` file extension to use for shared library if ``soname_pattern_base``
is not specified.
* ``static_suffix`` (default "a") file extension to use for static library.
* ``use_stack_protector`` (default "true") specify if by default stack smashing
protections should be enabled.
* ``uses_pkg_config`` (default "yes") specify if by default a pkg-config file
should be created.

20
lib/Botan-3.2.0/doc/dev_ref/contents.rst Normal file
View File

@@ -0,0 +1,20 @@
Developer Reference
=====================
This section contains information useful to people making
contributions to the library
.. toctree::
:maxdepth: 1
contributing
configure
test_framework
continuous_integration
fuzzing
release_process
todo
os
oids
reading_list
mistakes

52
lib/Botan-3.2.0/doc/dev_ref/continuous_integration.rst Normal file
View File

@@ -0,0 +1,52 @@
Continuous Integration and Automated Testing
===============================================
CI Build Script
----------------
The Github Actions builds are orchestrated using a script
``src/scripts/ci_build.py``. This allows one to easily reproduce the CI process
on a local machine.
Github Actions
---------------
https://github.com/randombit/botan/actions/workflows/ci.yml
Github Actions is the primary CI, and tests the Linux, Windows, macOS, and iOS
builds. Among other things it runs tests using valgrind, cross-compilation
for various architectures (currently including ARM and PPC64), MinGW build,
and a build that produces the coverage report.
The Github Actions configuration is in ``.github/workflows/ci.yml`` which
executes platform dependent setup scripts ``src/scripts/ci/setup_gh_actions.sh``
or ``src/scripts/ci/setup_gh_actions.ps1`` and ``.../setup_gh_actions_after_vcvars.ps1``
to install needed packages and detect certain platform specifics like compiler
cache locations.
Then ``src/scripts/ci_build.py`` is invoked to steer the actual build and test
runs.
Github Actions (nightly)
-------------------------
https://github.com/randombit/botan/actions/workflows/nightly.yml
Some checks are just too slow to include in the main CI builds. These
are instead delegated to a scheduled job that runs every night against
master.
Currently these checks include a full run of ``valgrind`` (the valgrind build in
CI only runs a subset of the tests), and a run of ``clang-tidy`` with all
warnings (that we are currently clean for) enabled. Each of these jobs takes
about an hour to run. In the main CI, we aim to have no job take more than
half an hour.
OSS-Fuzz
----------
https://github.com/google/oss-fuzz/
OSS-Fuzz is a distributed fuzzer run by Google. Every night, the fuzzer harnesses
in ``src/fuzzer`` are built and run on many machines, with any findings reported
to the developers via email.

299
lib/Botan-3.2.0/doc/dev_ref/contributing.rst Normal file
View File

@@ -0,0 +1,299 @@
Notes for New Contributors
===================================
Source Code Layout
-------------------------------------------------
Under ``src`` there are directories
* ``lib`` is the library itself, more on that below
* ``cli`` is the command line application ``botan``
* ``tests`` contain what you would expect. Input files go under ``tests/data``.
* ``python/botan3.py`` is the Python ctypes wrapper
* ``bogo_shim`` contains the shim binary and configuration for
`BoringSSL's TLS test suite <https://github.com/google/boringssl/tree/master/ssl/test>`_
* ``fuzzer`` contains fuzz targets for various modules of the library
* ``build-data`` contains files read by the configure script. For
example ``build-data/cc/gcc.txt`` describes various gcc options.
* ``examples`` contains usage examples used in the documentation.
* ``scripts`` contains misc scripts: install, distribution, various
codegen things. Scripts controlling CI go under ``scripts/ci``.
* ``configs`` contains configuration files tools like pylint
* ``editors`` contains configuration files for editors like vscode and emacs
Under ``doc`` one finds the sources of this documentation
Library Layout
----------------------------------------
Under ``src/lib`` are several directories
* ``asn1`` is the DER encoder/decoder
* ``base`` defines some high level types
* ``block`` contains the block cipher implementations
* ``codec`` has hex, base64, base32, base58
* ``compat`` a (partial) compatibility layer for the libsodium API
* ``compression`` has the compression wrappers (zlib, bzip2, lzma)
* ``entropy`` has various entropy sources used by some of the RNGs
* ``ffi`` is the C99 API
* ``filters`` is a filter/pipe API for data transforms
* ``hash`` contains the hash function implementations
* ``kdf`` contains the key derivation functions
* ``mac`` contains the message authentication codes
* ``math`` is the big integer math library. It is divided into three parts:
``mp`` which are the low level algorithms; ``bigint`` which is a C++ wrapper
around ``mp``, and ``numbertheory`` which contains higher level algorithms like
primality testing and exponentiation
* ``misc`` contains odds and ends: format preserving encryption, SRP, threshold
secret sharing, all or nothing transform, and others
* ``modes`` contains block cipher modes (CBC, GCM, etc)
* ``passhash`` contains password hashing algorithms for authentication
* ``pbkdf`` contains password hashing algorithms for key derivation
* ``pk_pad`` contains padding schemes for public key algorithms
* ``prov`` contains bindings to external libraries such as PKCS #11
* ``psk_db`` contains a generic interface for a Pre-Shared-Key database
* ``pubkey`` contains the public key algorithms
* ``rng`` contains the random number generators
* ``stream`` contains the stream ciphers
* ``tls`` contains the TLS implementation
* ``utils`` contains various utility functions and types
* ``x509`` is X.509 certificates, PKCS #10 requests, OCSP
Each of these folders can contain subfolders which are treated as modules if they
contain an ``info.txt`` file. These submodules have an implicit dependency on their
parent module. The chapter :ref:`configure_script` contains more information on
Botan's module architecture.
Sending patches
----------------------------------------
All contributions should be submitted as pull requests via GitHub
(https://github.com/randombit/botan). If you are planning a large
change, open a discussion ticket on github before starting out to make
sure you are on the right path. And once you have something written,
even if it is not complete/ready to go, feel free to open a draft PR
for early review and comment.
If possible please sign your git commits using a PGP key.
See https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work for
instructions on how to set this up.
Depending on what your change is, your PR should probably also include an update
to ``news.rst`` with a note explaining the change. If your change is a
simple bug fix, a one sentence description is perhaps sufficient. If there is an
existing ticket on GitHub with discussion or other information, reference it in
your change note as 'GH #000'.
Update ``doc/credits.txt`` with your information so people know what you did!
If you are interested in contributing but don't know where to start check out
``doc/dev_ref/todo.rst`` for some ideas - these are changes we would almost
certainly accept once they've passed code review.
Also, try building and testing it on whatever hardware you have handy,
especially unusual platforms, or using C++ compilers other than the regularly
tested GCC, Clang, and Visual Studio.
FFI Additions
----------------
If adding a new function declaration to ``ffi.h``, the same PR must also add the
same declaration in the Python binding ``botan3.py``, in addition the new API
functionality must be exposed to Python and a test written in Python.
Git Usage
----------------------------------------
Do *NOT* merge ``master`` into your topic branch, this creates needless commits
and noise in history. Instead, as needed, rebase your branch against master
(``git rebase -i master``) and force push the branch to update the PR. If the
GitHub PR page does not report any merge conflicts and nobody asks you to
rebase, you don't need to rebase.
Try to keep your history clean and use rebase to squash your commits as
needed. If your diff is less than roughly 100 lines, it should probably be a
single commit. Only split commits as needed to help with review/understanding of
the change.
Python
----------------------------------------
Scripts should be in Python 3 whenever possible.
For configure.py (and helper scripts install.py, cleanup.py and build_docs.py)
the target is stock (no modules outside the standard library) CPython 3.x.
Support for PyPy, etc is great when viable (in the sense of not causing problems
for 3.x, and not requiring huge blocks of version dependent code). As running
this program successfully is required for a working build, making it as portable
as possible is considered key.
The python wrapper botan3.py targets CPython 3.x, and latest PyPy. Note that
a single file is used to avoid dealing with any of Python's various crazy module
distribution issues.
For random scripts not typically run by an end-user (codegen, visualization, and
so on) there isn't any need to worry about platform independence. Here it's fine
to depend on any useful modules such as graphviz or matplotlib, regardless if it
is available from a stock CPython install.
Build Tools and Hints
----------------------------------------
If you don't already use it for all your C/C++ development, install ``ccache``
(or on Windows, ``sccache``) right now, and configure a large cache on a fast
disk. It allows for very quick rebuilds by caching the compiler output.
Use ``--enable-sanitizers=`` flag to enable various sanitizer checks. Supported
values including "address" and "undefined" for GCC and Clang. GCC also supports
"iterator" (checked iterators), and Clang supports "memory" (MSan) and
"coverage" (for fuzzing).
On Linux if you have the ``lcov`` and ``gcov`` tools installed, then running
``./src/scripts/ci_build.py coverage`` will produce a coverage enabled build,
run the tests, test the fuzzers against a corpus, and produce an HTML report
of total coverage. This coverage build requires the development headers for
zlib, bzip2, liblzma, TrouSerS (libtspi), and Sqlite3.
Copyright Notice
----------------------------------------
At the top of any new file add a comment with a copyright and a reference to the
license, for example::
/*
* (C) 202x <You>
*
* Botan is released under the Simplified BSD License (see license.txt)
*/
If you are making a substantial or non-trivial change to an existing file, add
or update your own copyright statement at the top of each file.
Style Conventions
----------------------------------------
When writing your code remember the need for it to be easily understood by
reviewers and auditors, both at the time of the patch submission and in the
future.
Avoid complicated template metaprogramming where possible. It has its places but
should be used judiciously.
When designing a new API (for use either by library users or just internally)
try writing out the calling code first. That is, write out some code calling
your idealized API, then just implement that API. This can often help avoid
cut-and-paste by creating the correct abstractions needed to solve the problem
at hand.
The C++11 ``auto`` keyword is very convenient but only use it when the type
truly is obvious (considering also the potential for unexpected integer
conversions and the like, such as an apparent uint8_t being promoted to an int).
Unless there is a specific reason otherwise (eg due to calling some C API which
requires exactly a ``long*`` be provided) integer types should be either
``(u)intXX_t`` or ``size_t``. If the variable is used for integer values of "no
particular size", as in the loop ``for(some_type i = 0; i != 100; ++i)`` then
the type should be ``size_t``. Use one of the specific size integer types only
when there is a algorithmic/protocol reason to use an integer of that size. For
example if a parsing a protocol that uses 16-bit integer fields to encode a
length, naturally one would use ``uint16_t`` there.
If a variable is defined and not modified, declare it ``const``. Some exception
for very short-lived variables, but generally speaking being able to read the
declaration and know it will not be modified is useful.
Use ``override`` annotations whenever overriding a virtual function. If
introducing a new type that is not intended for further derivation, mark it ``final``.
Avoid explicit ``new`` or (especially) explicit ``delete``: use RAII,
``make_unique``, etc.
Use ``m_`` prefix on all member variables.
``clang-format`` is used for all C++ formatting. The configuration is
in ``.clang-format`` in the root directory. You can rerun the
formatter using ``make fmt`` or by invoking the script
``src/scripts/dev_tools/run_clang_format.py``. If the output would be
truly horrible, it is allowed to disable formatting for a specific
area using ``// clang-format off`` annotations.
.. note::
Since the output of clang-format varies from version to version, we
currently require using exactly ``clang-format 15``.
Use braces on both sides of if/else blocks, even if only using a single
statement.
Avoid ``using namespace`` declarations, even inside of single functions. One
allowed exception is ``using namespace std::placeholders`` in functions which
use ``std::bind``. (But, don't use ``std::bind`` - use a lambda instead).
Use ``::`` to explicitly refer to the global namespace (eg, when calling an OS
or external library function like ``::select`` or ``::sqlite3_open``).
Use of External Dependencies
----------------------------------------
Compiler Dependencies
~~~~~~~~~~~~~~~~~~~~~~~
The library should always be as functional as possible when compiled with just
Standard C++20. However, feel free to use the full language.
Use of compiler extensions is fine whenever appropriate; this is typically
restricted to a single file or an internal header. Compiler extensions used
currently include native uint128_t, SIMD intrinsics, inline asm syntax and so
on, so there are some existing examples of appropriate use.
Generally intrinsics or inline asm is preferred over bare assembly to avoid
calling convention issues among different platforms; the improvement in
maintainability is seen as worth any potential performance tradeoff. One risk
with intrinsics is that the compiler might rewrite your clever const-time SIMD
into something with a conditional jump, but code intended to be const-time
should in any case be annotated (using ``CT::poison``) so it can be checked at
runtime with tools.
Operating System Dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you're adding a small OS dependency in some larger piece of code, try to
contain the actual non-portable operations to utils/os_utils.* and then call
them from there.
As a policy, operating systems which are not supported by their original vendor
are not supported by Botan either. Patches that complicate the code in order to
support obsolete operating systems will likely be rejected. In writing OS
specific code, feel free to assume roughly POSIX 2008, or for Windows, Windows 8
/Server 2012 (which are as of this writing the oldest versions still supported
by Microsoft).
Some operating systems, such as OpenBSD, only support the latest release. For
such cases, it's acceptable to add code that requires APIs added in the most
recent release of that OS as soon as the release is available.
Library Dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Any external library dependency - even optional ones - is met with as one PR
submitter put it "great skepticism".
At every API boundary there is potential for confusion that does not exist when
the call stack is all contained within the boundary. So the additional API
really needs to pull its weight. For example a simple text parser or such which
can be trivially implemented is not really for consideration. As a rough idea of
the bar, equate the viewed cost of an external dependency as at least 1000
additional lines of code in the library. That is, if the library really does
need this functionality, and it can be done in the library for less than that,
then it makes sense to just write the code. Yup.
Currently the (optional) external dependencies of the library are several
compression libraries (zlib, bzip2, lzma), sqlite3 database, Trousers (TPM
integration), plus various operating system utilities like basic filesystem
operations. These provide major pieces of functionality which seem worth the
trouble of maintaining an integration with.
At this point the most plausible examples of an appropriate new external
dependency are all deeper integrations with system level cryptographic
interfaces (CommonCrypto, CryptoAPI, /dev/crypto, iOS keychain, TPM 2.0, etc)

91
lib/Botan-3.2.0/doc/dev_ref/fuzzing.rst Normal file
View File

@@ -0,0 +1,91 @@
Fuzzing The Library
============================
Botan comes with a set of fuzzing endpoints which can be used to test
the library.
.. highlight:: shell
Fuzzing with libFuzzer
------------------------
To fuzz with libFuzzer (https://llvm.org/docs/LibFuzzer.html), you'll first
need to compile libFuzzer::
$ svn co https://llvm.org/svn/llvm-project/compiler-rt/trunk/lib/fuzzer libFuzzer
$ cd libFuzzer && clang -c -g -O2 -std=c++11 *.cpp
$ ar cr libFuzzer.a libFuzzer/*.o
Then build the fuzzers::
$ ./configure.py --cc=clang --build-fuzzer=libfuzzer --unsafe-fuzzer-mode \
--with-debug-info --enable-sanitizers=coverage,address,undefined
$ make fuzzers
Enabling 'coverage' sanitizer flags is required for libFuzzer to work.
Address sanitizer and undefined sanitizer are optional.
The fuzzer binaries will be in `build/fuzzer`. Simply pick one and run it, optionally
also passing a directory containing corpus inputs.
Using `libfuzzer` build mode implicitly assumes the fuzzers need to
link with `libFuzzer`; if another library is needed (for example in
OSS-Fuzz, which uses `libFuzzingEngine`), use the flag
`--with-fuzzer-lib` to specify the desired name.
Fuzzing with AFL
--------------------
To fuzz with AFL (http://lcamtuf.coredump.cx/afl/)::
$ ./configure.py --with-sanitizers --build-fuzzer=afl --unsafe-fuzzer-mode --cc-bin=afl-g++
$ make fuzzers
For AFL sanitizers are optional. You can also use `afl-clang-fast++`
or `afl-clang++`, be sure to set `--cc=clang` also.
The fuzzer binaries will be in `build/fuzzer`. To run them you need to
run under `afl-fuzz`::
$ afl-fuzz -i corpus_path -o output_path ./build/fuzzer/binary
Fuzzing with TLS-Attacker
--------------------------
TLS-Attacker (https://github.com/RUB-NDS/TLS-Attacker) includes a mode for fuzzing
TLS servers. A prebuilt copy of TLS-Attacker is available in a git repository::
$ git clone --depth 1 https://github.com/randombit/botan-ci-tools.git
To run it against Botan's server::
$ ./configure.py --with-sanitizers
$ make botan
$ ./src/scripts/run_tls_attacker.py ./botan ./botan-ci-tools
Output and logs from the fuzzer are placed into `/tmp`. See the
TLS-Attacker documentation for more information about how to use this
tool.
Input Corpus
-----------------------
AFL requires an input corpus, and libFuzzer can certainly make good
use of it.
Some crypto corpus repositories include
* https://github.com/randombit/crypto-corpus
* https://github.com/mozilla/nss-fuzzing-corpus
* https://github.com/google/boringssl/tree/master/fuzz
* https://github.com/openssl/openssl/tree/master/fuzz/corpora
Adding new fuzzers
---------------------
New fuzzers are created by adding a source file to `src/fuzzers` which
have the signature:
``void fuzz(const uint8_t in[], size_t len)``
After adding your fuzzer, rerun ``./configure.py`` and build.

79
lib/Botan-3.2.0/doc/dev_ref/mistakes.rst Normal file
View File

@@ -0,0 +1,79 @@
Mistakes Were Made
===================
These are mistakes made early on in the project's history which are difficult to
fix now, but mentioned in the hope they may serve as an example for others.
C++ API
---------
As an implementation language, I still think C++ is the best choice (or at least
the best choice available in early '00s) at offering good performance,
reasonable abstractions, and low overhead. But the user API should have been
pure C with opaque structs (rather like the FFI layer, which was added much
later). Then an expressive C++ API could be built on top of the C API. This
would have given us a stable ABI, allowed C applications to use the library, and
(these days) make it easier to progressively rewrite the library in Rust.
Public Algorithm Specific Classes
------------------------------------
Classes like AES_128 and SHA_256 should never have been exposed to applications.
Intead such operations should have been accessible only via the higher level
interfaces (here BlockCipher and HashFunction). This would substantially reduce
the overall API and ABI surface.
[These interfaces were made internal in 3.0]
Header Directories
-------------------
It would have been better to install all headers as ``X/header.h``
where ``X`` is the base dir in the source, eg ``block/aes128.h``,
``hash/md5.h``, ...
Exceptions
-----------
Constant ABI headaches from this, and it impacts performance and makes APIs
harder to understand. Should have been handled with a result<> type instead.
Alternatively, and possibly more practically, there should have not been any
exception hierarchy (or at least not one visible to users) - instead only the
high level Exception type with contains an error type enum.
Virtual inheritance
---------------------
This was used in the public key interfaces and the hierarchy is a tangle.
Public and private keys should be distinct classes, with a function on private
keys that creates a new object corresponding to the public key.
Cipher Interface
------------------
The cipher interface taking a secure_vector that it reads from and writes to was
an artifact of an earlier design which supported both compression and encryption
in a single API. But it leads to inefficient copies.
(I am hoping this issue can be somewhat fixed by introducing a new cipher API
and implementing the old API in terms of the new one.)
Pipe Interface
----------------
On the surface this API seems very convenient and easy to use. And it is. But
the downside is it makes the application code totally opaque; some bytes go into
a Pipe object and then come out the end transformed in some way. What happens in
between? Unless the Pipe was built in the same function and you can see the
parameters to the constructor, there is no way to find out.
The problems with the Pipe API are documented, and it is no longer used within
the library itself. But since many people seem to like it and many applications
use it, we are stuck at least with maintaining it as it currently exists.
License
---------
MIT is more widely used and doesn't have the ambiguity surrounding the
various flavors of BSD.

92
lib/Botan-3.2.0/doc/dev_ref/oids.rst Normal file
View File

@@ -0,0 +1,92 @@
Private OID Assignments
==========================
The library uses some OIDs under a private arc assigned by IANA,
1.3.6.1.4.1.25258
Values currently assigned are::
randombit OBJECT IDENTIFIER ::= { 1 3 6 1 4 1 25258 }
publicKey OBJECT IDENTIFIER ::= { randombit 1 }
mceliece OBJECT IDENTIFIER ::= { publicKey 3 }
-- { publicKey 4 } previously used as private X25519
-- { publicKey 5 } previously used for XMSS draft 6
gost-3410-with-sha256 OBJECT IDENTIFIER ::= { publicKey 6 1 }
kyber OBJECT IDENTIFIER ::= { publicKey 7 }
kyber-90s OBJECT IDENTIFIER ::= { publicKey 11 }
kyber-512 OBJECT IDENTIFIER ::= { kyber 1 }
kyber-768 OBJECT IDENTIFIER ::= { kyber 2 }
kyber-1024 OBJECT IDENTIFIER ::= { kyber 3 }
kyber-512-90s OBJECT IDENTIFIER ::= { kyber-90s 1 }
kyber-768-90s OBJECT IDENTIFIER ::= { kyber-90s 2 }
kyber-1024-90s OBJECT IDENTIFIER ::= { kyber-90s 3 }
xmss OBJECT IDENTIFIER ::= { publicKey 8 }
-- The current dilithium implementation is compatible with the reference
-- implementation commit 3e9b9f1412f6c7435dbeb4e10692ea58f181ee51
dilithium OBJECT IDENTIFIER ::= { publicKey 9 }
dilithium-aes OBJECT IDENTIFIER ::= { publicKey 10 }
dilithium-4x4 OBJECT IDENTIFIER ::= { dilithium 1 }
dilithium-6x5 OBJECT IDENTIFIER ::= { dilithium 2 }
dilithium-8x7 OBJECT IDENTIFIER ::= { dilithium 3 }
dilithium-aes-4x4 OBJECT IDENTIFIER ::= { dilithium-aes 1 }
dilithium-aes-6x5 OBJECT IDENTIFIER ::= { dilithium-aes 2 }
dilithium-aes-8x7 OBJECT IDENTIFIER ::= { dilithium-aes 3 }
SphincsPlus OBJECT IDENTIFIER ::= { publicKey 12 }
SphincsPlus-shake OBJECT IDENTIFIER ::= { SphincsPlus 1 }
SphincsPlus-sha2 OBJECT IDENTIFIER ::= { SphincsPlus 2 }
SphincsPlus-haraka OBJECT IDENTIFIER ::= { SphincsPlus 3 }
SphincsPlus-shake-128s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-shake256 1 }
SphincsPlus-shake-128f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-shake256 2 }
SphincsPlus-shake-192s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-shake256 3 }
SphincsPlus-shake-192f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-shake256 4 }
SphincsPlus-shake-256s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-shake256 5 }
SphincsPlus-shake-256f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-shake256 6 }
SphincsPlus-sha2-128s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-sha256 1 }
SphincsPlus-sha2-128f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-sha256 2 }
SphincsPlus-sha2-192s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-sha256 3 }
SphincsPlus-sha2-192f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-sha256 4 }
SphincsPlus-sha2-256s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-sha256 5 }
SphincsPlus-sha2-256f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-sha256 6 }
SphincsPlus-haraka-128s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-haraka 1 }
SphincsPlus-haraka-128f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-haraka 2 }
SphincsPlus-haraka-192s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-haraka 3 }
SphincsPlus-haraka-192f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-haraka 4 }
SphincsPlus-haraka-256s-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-haraka 5 }
SphincsPlus-haraka-256f-r3.1 OBJECT IDENTIFIER ::= { SphincsPlus-haraka 6 }
symmetricKey OBJECT IDENTIFIER ::= { randombit 3 }
ocbModes OBJECT IDENTIFIER ::= { symmetricKey 2 }
aes-128-ocb OBJECT IDENTIFIER ::= { ocbModes 1 }
aes-192-ocb OBJECT IDENTIFIER ::= { ocbModes 2 }
aes-256-ocb OBJECT IDENTIFIER ::= { ocbModes 3 }
serpent-256-ocb OBJECT IDENTIFIER ::= { ocbModes 4 }
twofish-256-ocb OBJECT IDENTIFIER ::= { ocbModes 5 }
camellia-128-ocb OBJECT IDENTIFIER ::= { ocbModes 6 }
camellia-192-ocb OBJECT IDENTIFIER ::= { ocbModes 7 }
camellia-256-ocb OBJECT IDENTIFIER ::= { ocbModes 8 }
sivModes OBJECT IDENTIFIER ::= { symmetricKey 4 }
aes-128-siv OBJECT IDENTIFIER ::= { sivModes 1 }
aes-192-siv OBJECT IDENTIFIER ::= { sivModes 2 }
aes-256-siv OBJECT IDENTIFIER ::= { sivModes 3 }
serpent-256-siv OBJECT IDENTIFIER ::= { sivModes 4 }
twofish-256-siv OBJECT IDENTIFIER ::= { sivModes 5 }
camellia-128-siv OBJECT IDENTIFIER ::= { sivModes 6 }
camellia-192-siv OBJECT IDENTIFIER ::= { sivModes 7 }
camellia-256-siv OBJECT IDENTIFIER ::= { sivModes 8 }
sm4-128-siv OBJECT IDENTIFIER ::= { sivModes 9 }

70
lib/Botan-3.2.0/doc/dev_ref/os.rst Normal file
View File

@@ -0,0 +1,70 @@
OS Features
========================================
A summary of OS features as defined in ``src/build-data/os``.
::
a: aix
a: android
c: cygwin
d: dragonfly
e: emscripten
f: freebsd
g: generic
h: haiku
h: hpux
h: hurd
i: ios
l: linux
l: llvm
m: macos
m: mingw
n: netbsd
n: none
o: openbsd
q: qnx
s: solaris
u: uwp
w: windows
.. csv-table::
:header: "Feature", "a", "a", "c", "d", "e", "f", "g", "h", "h", "h", "i", "l", "l", "m", "m", "n", "n", "o", "q", "s", "u", "w"
"alloc_conceal", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " "
"apple_keychain", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " "
"arc4random", " ", "X", " ", "X", " ", "X", " ", " ", " ", " ", "X", " ", " ", "X", " ", "X", " ", "X", " ", " ", " ", " "
"atomics", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", " ", "X", "X", "X", "X", "X"
"auxinfo", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " "
"cap_enter", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " "
"ccrandom", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " "
"certificate_store", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", "X"
"clock_gettime", "X", "X", " ", "X", " ", "X", " ", "X", "X", "X", " ", "X", " ", "X", " ", "X", " ", "X", "X", "X", " ", " "
"commoncrypto", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " "
"crypto_ng", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " "
"dev_random", "X", "X", "X", "X", "X", "X", " ", "X", "X", "X", " ", "X", " ", "X", " ", "X", " ", "X", "X", "X", " ", " "
"elf_aux_info", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " "
"explicit_bzero", " ", " ", " ", "X", " ", "X", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", "X", " ", " ", " ", " "
"explicit_memset", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " "
"filesystem", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", "X", " ", "X", "X", "X", "X", "X"
"getauxval", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", " "
"getentropy", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", "X", " ", "X", " ", " ", " ", "X", " ", "X", " ", " "
"getrandom", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", " "
"pledge", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " "
"posix1", "X", "X", "X", "X", "X", "X", " ", "X", "X", "X", "X", "X", " ", "X", " ", "X", " ", "X", "X", "X", " ", " "
"posix_mlock", "X", "X", " ", "X", " ", "X", " ", " ", "X", "X", "X", "X", " ", "X", " ", "X", " ", "X", "X", "X", " ", " "
"prctl", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " ", " ", " "
"proc_fs", "X", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", "X", " ", " "
"rtlgenrandom", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", "X"
"rtlsecurezeromemory", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", "X"
"sandbox_proc", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", " ", " "
"setppriv", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " "
"sockets", "X", "X", "X", "X", " ", "X", " ", "X", "X", "X", "X", "X", " ", "X", " ", "X", " ", "X", "X", "X", " ", " "
"thread_local", "X", "X", "X", "X", " ", "X", "X", "X", "X", "X", "X", "X", " ", "X", "X", "X", " ", "X", "X", "X", "X", "X"
"threads", "X", "X", "X", "X", " ", "X", "X", "X", "X", "X", "X", "X", " ", "X", "X", "X", " ", "X", "X", "X", "X", "X"
"virtual_lock", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", " ", "X"
"win32", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", " ", " ", " ", " ", " ", "X", "X"
"winsock2", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", " ", "X", "X"
.. note::
This file is auto generated by ``src/scripts/gen_os_features.py``. Dont modify it manually.

93
lib/Botan-3.2.0/doc/dev_ref/reading_list.rst Normal file
View File

@@ -0,0 +1,93 @@
Reading List
================
These are papers, articles and books that are interesting or useful from the
perspective of crypto implementation.
Papers
--------
Implementation Techniques
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* "Randomizing the Montgomery Powering Ladder"
Le, Tan, Tunstall https://eprint.iacr.org/2015/657
A variant of Algorithm 7 is used for GF(p) point multplications when
BOTAN_POINTGFP_BLINDED_MULTIPLY_USE_MONTGOMERY_LADDER is set
* "Accelerating AES with vector permute instructions"
Mike Hamburg https://shiftleft.org/papers/vector_aes/
His public doman assembly code was rewritten into SSS3 intrinsics
for aes_ssse3.
* "Elliptic curves and their implementation" Langley
http://www.imperialviolet.org/2010/12/04/ecc.html
Describes sparse representations for ECC math
Random Number Generation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* "On Extract-then-Expand Key Derivation Functions and an HMAC-based KDF"
Hugo Krawczyk http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.131.8254
RNG design underlying HMAC_RNG
AES Side Channels
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* "Software mitigations to hedge AES against cache-based software side
channel vulnerabilities" https://eprint.iacr.org/2006/052.pdf
* "Cache Games - Bringing Access-Based Cache Attacks on AES to Practice"
http://www.ieee-security.org/TC/SP2011/PAPERS/2011/paper031.pdf
* "Cache-Collision Timing Attacks Against AES" Bonneau, Mironov
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.88.4753
Public Key Side Channels
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* "Fast Elliptic Curve Multiplications Resistant against Side Channel Attacks"
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.98.1028&rep=rep1&type=pdf
* "Resistance against Differential Power Analysis for Elliptic Curve Cryptosystems"
Coron http://www.jscoron.fr/publications/dpaecc.pdf
* "Further Results and Considerations on Side Channel Attacks on RSA"
Klima, Rosa https://eprint.iacr.org/2002/071
Side channel attacks on RSA-KEM and MGF1-SHA1
* "Side-Channel Attacks on the McEliece and Niederreiter Public-Key Cryptosystems"
Avanzi, Hoerder, Page, and Tunstall https://eprint.iacr.org/2010/479
* "Minimum Requirements for Evaluating Side-Channel Attack Resistance
of Elliptic Curve Implementations" BSI
https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Zertifizierung/Interpretationen/AIS_46_ECCGuide_e_pdf.pdf
Books
------
* "Handbook of Elliptic and Hyperelliptic Curve Cryptography"
Cohen and Frey https://www.hyperelliptic.org/HEHCC/
An excellent reference for ECC math, algorithms, and side channels
* "Post-Quantum Cryptography" Bernstein, Buchmann, Dahmen
Covers code, lattice, and hash based cryptography
Standards
-----------
* IEEE 1363 http://grouper.ieee.org/groups/1363/
Very influential early in the library lifetime, so a lot of terminology used
in the public key (such as "EME" for message encoding) code comes from here.
* ISO/IEC 18033-2 http://www.shoup.net/iso/std4.pdf
RSA-KEM, PSEC-KEM
* NIST SP 800-108
http://csrc.nist.gov/publications/nistpubs/800-108/sp800-108.pdf
KDF schemes
* NIST SP 800-90A
http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
HMAC_DRBG, Hash_DRBG, CTR_DRBG, maybe one other thing?

103
lib/Botan-3.2.0/doc/dev_ref/release_process.rst Normal file
View File

@@ -0,0 +1,103 @@
Release Process and Checklist
========================================
Releases are done quarterly, normally on the second non-holiday Monday
of January, April, July and October. A feature freeze goes into effect
starting 9 days before the release.
.. highlight:: shell
.. note::
This information is only useful if you are a developer of botan who
is creating a new release of the library.
Pre Release Testing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Do maintainer-mode builds with Clang and GCC to catch any warnings
that should be corrected.
Other checks which are not in CI:
- Native compile on FreeBSD x86-64
- Native compile on at least one unusual platform (AIX, NetBSD, ...)
- Build the website content to detect any Doxygen problems
- Test many build configurations (using `src/scripts/test_all_configs.py`)
- Build/test SoftHSM
Confirm that the release notes in ``news.rst`` are accurate and
complete and that the version number in ``version.txt`` is correct.
Tag the Release
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Update the release date in the release notes and change the entry for
the appropriate branch in ``readme.rst`` to point to the new release.
Now check in, and backport changes to the release branch::
$ git commit readme.rst news.rst -m "Update for 3.8.2 release"
$ git checkout release-3
$ git merge master
$ git tag 3.8.2
Build The Release Tarballs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The release script is ``src/scripts/dist.py`` and must be run from a
git workspace.
$ src/scripts/dist.py 3.8.2
One useful option is ``--output-dir``, which specifies where the
output will be placed.
Now do a final build/test of the released tarball.
The ``--pgp-key-id`` option is used to specify a PGP keyid. If set,
the script assumes that it can execute GnuPG and will attempt to
create signatures for the tarballs. The default value is ``EFBADFBC``,
which is the official signing key. You can use ``--pgp-key-id=none``
to avoid creating any signature, though official distributed releases
*should not* be released without signatures.
The releases served on the official site are taken from the contents
in a git repository::
$ git checkout git@botan.randombit.net:/srv/git/botan-releases.git
$ src/scripts/dist.py 3.8.2 --output-dir=botan-releases
$ cd botan-releases
$ sha256sum Botan-3.8.2.tgz >> sha256sums.txt
$ git add .
$ git commit -m "Release version 3.8.2"
$ git push origin master
A cron job updates the live site every 10 minutes.
Push to GitHub
^^^^^^^^^^^^^^^^^^
Don't forget to also push tags::
$ git push origin --tags release-3 master
Update The Website
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The website content is created by ``src/scripts/website.py``.
The website is mirrored automatically from a git repository which must be updated::
$ git checkout git@botan.randombit.net:/srv/git/botan-website.git
$ ./src/scripts/website.py --output botan-website
$ cd botan-website
$ git add .
$ git commit -m "Update for 3.8.2"
$ git push origin master
Announce The Release
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Send an email to the botan-announce and botan-devel mailing lists
noting that a new release is available.

314
lib/Botan-3.2.0/doc/dev_ref/test_framework.rst Normal file
View File

@@ -0,0 +1,314 @@
Test Framework
================
Botan uses a custom-built test framework. Some portions of it are
quite similar to assertion-based test frameworks such as Catch or
Gtest, but it also includes many features which are well suited for
testing cryptographic algorithms.
The intent is that the test framework and the test suite evolve
symbiotically; as a general rule of thumb if a new function would make
the implementation of just two distinct tests simpler, it is worth
adding to the framework on the assumption it will prove useful again.
Feel free to propose changes to the test system.
When writing a new test, there are three key classes that are used,
namely ``Test``, ``Test::Result``, and ``Text_Based_Test``. A ``Test``
(or ``Test_Based_Test``) runs and returns one or more ``Test::Result``.
Namespaces in Test
-------------------
The test code lives in a distinct namespace (``Botan_Tests``) and all
code in the tests which calls into the library should use the
namespace prefix ``Botan::`` rather than a ``using namespace``
declaration. This makes it easier to see where the test is actually
invoking the library, and makes it easier to reuse test code for
applications.
Test Data
-----------
The test framework is heavily data driven. As of this writing, there
is about 1 Mib of test code and 17 MiB of test data. For most (though
certainly not all) tests, it is better to add a data file representing
the input and outputs, and run the tests over it. Data driven tests
make adding or editing tests easier, for example by writing scripts
which produce new test data and output it in the expected format.
Test
--------
.. cpp:class:: Test
.. cpp:function:: virtual std::vector<Test::Result> run() = 0
This is the key function of a ``Test``: it executes and returns a
list of results. Almost all other functions on ``Test`` are
static functions which just serve as helper functions for ``run``.
.. cpp:function:: static std::string read_data_file(const std::string& path)
Return the contents of a data file and return it as a string.
.. cpp:function:: static std::vector<uint8_t> read_binary_data_file(const std::string& path)
Return the contents of a data file and return it as a vector of
bytes.
.. cpp:function:: static std::string data_file(const std::string& what)
An alternative to ``read_data_file`` and ``read_binary_file``,
use only as a last result, typically for library APIs which
themselves accept a filename rather than a data blob.
.. cpp:function:: static bool run_long_tests() const
Returns true if the user gave option ``--run-long-tests``. Use
this to gate particularly time-intensive tests.
.. cpp:function:: static Botan::RandomNumberGenerator& rng()
Returns a reference to a fast, not cryptographically secure
random number generator. It is deterministicly seeded with the
seed logged by the test runner, so it is possible to reproduce
results in "random" tests.
Tests are registered using the macro ``BOTAN_REGISTER_TEST`` which
takes 2 arguments: the name of the test and the name of the test class.
For example given a ``Test`` instance named ``MyTest``, use::
BOTAN_REGISTER_TEST("mytest", MyTest);
All test names should contain only lowercase letters, numbers, and
underscore.
Test::Result
-------------
.. cpp:class:: Test::Result
A ``Test::Result`` records one or more tests on a particular topic
(say "AES-128/CBC" or "ASN.1 date parsing"). Most of the test functions
return true or false if the test was successful or not; this allows
performing conditional blocks as a result of earlier tests::
if(result.test_eq("first value", produced, expected))
{
// further tests that rely on the initial test being correct
}
Only the most commonly used functions on ``Test::Result`` are documented here,
see the header ``tests.h`` for more.
.. cpp:function:: Test::Result(const std::string& who)
Create a test report on a particular topic. This will be displayed in the
test results.
.. cpp:function:: bool test_success()
Report a test that was successful.
.. cpp:function:: bool test_success(const std::string& note)
Report a test that was successful, including some comment.
.. cpp:function:: bool test_failure(const std::string& err)
Report a test failure of some kind. The error string will be logged.
.. cpp:function:: bool test_failure(const std::string& what, const std::string& error)
Report a test failure of some kind, with a description of what failed and
what the error was.
.. cpp:function:: void test_failure(const std::string& what, const uint8_t buf[], size_t buf_len)
Report a test failure due to some particular input, which is provided as
arguments. Normally this is only used if the test was using some
randomized input which unexpectedly failed, since if the input is
hardcoded or from a file it is easier to just reference the test number.
.. cpp:function:: bool test_eq(const std::string& what, const std::string& produced, const std::string& expected)
Compare to strings for equality.
.. cpp:function:: bool test_ne(const std::string& what, const std::string& produced, const std::string& expected)
Compare to strings for non-equality.
.. cpp:function:: bool test_eq(const char* producer, const std::string& what, \
const uint8_t produced[], size_t produced_len, \
const uint8_t expected[], size_t expected_len)
Compare two arrays for equality.
.. cpp:function:: bool test_ne(const char* producer, const std::string& what, \
const uint8_t produced[], size_t produced_len, \
const uint8_t expected[], size_t expected_len)
Compare two arrays for non-equality.
.. cpp:function:: bool test_eq(const std::string& producer, const std::string& what, \
const std::vector<uint8_t>& produced, \
const std::vector<uint8_t>& expected)
Compare two vectors for equality.
.. cpp:function:: bool test_ne(const std::string& producer, const std::string& what, \
const std::vector<uint8_t>& produced, \
const std::vector<uint8_t>& expected)
Compare two vectors for non-equality.
.. cpp:function:: bool confirm(const std::string& what, bool expr)
Test that some expression evaluates to ``true``.
.. cpp:function:: template<typename T> bool test_not_null(const std::string& what, T* ptr)
Verify that the pointer is not null.
.. cpp:function:: bool test_lt(const std::string& what, size_t produced, size_t expected)
Test that ``produced`` < ``expected``.
.. cpp:function:: bool test_lte(const std::string& what, size_t produced, size_t expected)
Test that ``produced`` <= ``expected``.
.. cpp:function:: bool test_gt(const std::string& what, size_t produced, size_t expected)
Test that ``produced`` > ``expected``.
.. cpp:function:: bool test_gte(const std::string& what, size_t produced, size_t expected)
Test that ``produced`` >= ``expected``.
.. cpp:function:: bool test_throws(const std::string& what, std::function<void ()> fn)
Call a function and verify it throws an exception of some kind.
.. cpp:function:: bool test_throws(const std::string& what, const std::string& expected, std::function<void ()> fn)
Call a function and verify it throws an exception of some kind
and that the exception message exactly equals ``expected``.
Text_Based_Test
-----------------
A ``Text_Based_Text`` runs tests that are produced from a text file
with a particular format which looks somewhat like an INI-file::
# Comments begin with # and continue to end of line
[Header]
# Test 1
Key1 = Value1
Key2 = Value2
# Test 2
Key1 = Value1
Key2 = Value2
.. cpp:class:: VarMap
An object of this type is passed to each invocation of the text-based test.
It is used to access the test variables. All access takes a key, which is
one of the strings which was passed to the constructor of ``Text_Based_Text``.
Accesses are either required (``get_req_foo``), in which case an exception is
throwing if the key is not set, or optional (``get_opt_foo``) in which case
the test provides a default value which is returned if the key was not set
for this particular instance of the test.
.. cpp:function:: std::vector<uint8_t> get_req_bin(const std::string& key) const
Return a required binary string. The input is assumed to be hex encoded.
.. cpp:function:: std::vector<uint8_t> get_opt_bin(const std::string& key) const
Return an optional binary string. The input is assumed to be hex encoded.
.. cpp:function:: std::vector<std::vector<uint8_t>> get_req_bin_list(const std::string& key) const
.. cpp:function:: Botan::BigInt get_req_bn(const std::string& key) const
Return a required BigInt. The input can be decimal or (with "0x" prefix) hex encoded.
.. cpp:function:: Botan::BigInt get_opt_bn(const std::string& key, const Botan::BigInt& def_value) const
Return an optional BigInt. The input can be decimal or (with "0x" prefix) hex encoded.
.. cpp:function:: std::string get_req_str(const std::string& key) const
Return a required text string.
.. cpp:function:: std::string get_opt_str(const std::string& key, const std::string& def_value) const
Return an optional text string.
.. cpp:function:: size_t get_req_sz(const std::string& key) const
Return a required integer. The input should be decimal.
.. cpp:function:: size_t get_opt_sz(const std::string& key, const size_t def_value) const
Return an optional integer. The input should be decimal.
.. cpp:class:: Text_Based_Test : public Test
.. cpp:function:: Text_Based_Test(const std::string& input_file, \
const std::string& required_keys, \
const std::string& optional_keys = "")
This constructor is
.. note::
The final element of required_keys is the "output key", that is
the key which signifies the boundary between one test and the next.
When this key is seen, ``run_one_test`` will be invoked. In the
test input file, this key must always appear least for any particular
test. All the other keys may appear in any order.
.. cpp:function:: Test::Result run_one_test(const std::string& header, \
const VarMap& vars)
Runs a single test and returns the result of it. The ``header``
parameter gives the value (if any) set in a ``[Header]`` block.
This can be useful to distinguish several types of tests within a
single file, for example "[Valid]" and "[Invalid]".
.. cpp:function:: bool clear_between_callbacks() const
By default this function returns ``false``. If it returns
``true``, then when processing the data in the file, variables
are not cleared between tests. This can be useful when several
tests all use some common parameters.
Test Runner
-------------
If you are simply writing a new test there should be no need to modify
the runner, however it can be useful to be aware of its abilities.
The runner can run tests concurrently across many cores. By default single
threaded execution is used, but you can use ``--test-threads`` option to
specify the number of threads to use. If you use ``--test-threads=0`` then
the runner will probe the number of active CPUs and use that (but limited
to at most 16). If you want to run across many cores on a large machine,
explicitly specify a thread count. The speedup is close to linear.
The RNG used in the tests is deterministic, and the seed is logged for each
execution. You can cause the random sequence to repeat using ``--drbg-seed``
option.
.. note::
Currently the RNG is seeded just once at the start of execution. So you
must run the exact same sequence of tests as the original test run in
order to get reproducible results.
If you are trying to track down a bug that happens only occasionally, two very
useful options are ``--test-runs`` and ``--abort-on-first-fail``. The first
takes an integer and runs the specified test cases that many times. The second
causes abort to be called on the very first failed test. This is sometimes
useful when tracing a memory corruption bug.

160
lib/Botan-3.2.0/doc/dev_ref/todo.rst Normal file
View File

@@ -0,0 +1,160 @@
Todo List
========================================
Feel free to take one of these on if it interests you. Before starting
out on something, send an email to the dev list or open a discussion
ticket on GitHub to make sure you're on the right track.
Request a new feature by opening a pull request to update this file.
New Ciphers/Hashes/MACs
----------------------------------------
* GCM-SIV (RFC 8452)
* EME* tweakable block cipher (https://eprint.iacr.org/2004/125)
* PMAC
* SIV-PMAC
* Threefish-1024
* Skein-MAC
* FFX format preserving encryption (NIST 800-38G)
* Adiantum (https://eprint.iacr.org/2018/720)
* HPKE (draft-irtf-cfrg-hpke)
* Blake3
Improved Ciphers Implementations
----------------------------------------
* Stiched AES/GCM mode for CPUs supporting both AES and CLMUL
* Combine AES-NI, ARMv8 and POWER AES implementations (as already done for CLMUL)
* Support for VAES (Zen4/Ice Lake)
* NEON/VMX support for the SIMD based GHASH
* Vector permute AES only supports little-endian systems; fix for big-endian
* SM4 using AES-NI (https://github.com/mjosaarinen/sm4ni) or vector permute
* Poly1305 using AVX2
* SHA-512 using BMI2+AVX2
* Constant time bitsliced DES
* SIMD evaluation of SHA-2 and SHA-3 compression functions
* Improved Salsa implementations (SIMD_4x32 and/or AVX2)
* Add CLMUL/PMULL implementations for CRC24/CRC32
Public Key Crypto, Math
----------------------------------------
* Short vector optimization for BigInt
* Abstract representation of ECC point elements to allow specific
implementations of the field arithmetic depending upon the curve.
* Curves for pairings (BLS12-381)
* Identity based encryption
* Paillier homomorphic cryptosystem
* New PAKEs (pending CFRG bakeoff results)
* New post quantum schemes (pending NIST contest results)
* SPHINX password store (https://eprint.iacr.org/2018/695)
* X448 and Ed448
Utility Functions
------------------
* Make Memory_Pool more concurrent (currently uses a global lock)
* Guarded integer type to prevent overflow bugs
External Providers, Hardware Support
----------------------------------------
* Add support ARMv8.4-A SHA-512, SHA-3, SM3 and RNG
* Aarch64 inline asm for BigInt
* /dev/crypto provider (ciphers, hashes)
* Windows CryptoNG provider (ciphers, hashes)
* Extend Apple CommonCrypto provider (HMAC, CMAC, RSA, ECDSA, ECDH)
* Add support for iOS keychain access
* POWER8 SHA-2 extensions (GH #1486 + #1487)
* Add support VPSUM on big-endian PPC64 (GH #2252)
* Better TPM support: NVRAM, PCR measurements, sealing
* Add support for TPM 2.0 hardware
TLS
----------------------------------------
* Make DTLS support optional at build time
* Improve/optimize DTLS defragmentation and retransmission
* Make RSA optional at build time
* Make finite field DH optional at build time
* Certificate Transparency extensions
* TLS supplemental authorization data (RFC 4680, RFC 5878)
* DTLS-SCTP (RFC 6083)
PKIX
----------------------------------------
* Further tests of validation API (see GH #785)
* Test suite for validation of 'real world' cert chains (GH #611)
* X.509 policy constraints
* OCSP responder logic
New Protocols / Formats
----------------------------------------
* Noise protocol
* ACME protocol
* Cryptographic Message Syntax (RFC 5652)
* Fernet symmetric encryption (https://cryptography.io/en/latest/fernet/)
* RNCryptor format (https://github.com/RNCryptor/RNCryptor)
* Age format (https://age-encryption.org/v1)
* Useful OpenPGP subset 1: symmetrically encrypted files.
Not aiming to process arbitrary OpenPGP, but rather produce
something that happens to be readable by `gpg` and is relatively
simple to process for decryption. Require AEAD mode (EAX/OCB).
* Useful OpenPGP subset 2: Process OpenPGP public keys
* Useful OpenPGP subset 3: Verification of OpenPGP signatures
Cleanups
-----------
* Unicode path support on Windows (GH #1615)
* The X.509 path validation tests have much duplicated logic
New C APIs
----------------------------------------
* PKCS10 requests
* Certificate signing
* CRLs
* Expose TLS
* Expose NIST key wrap with padding
* Expose secret sharing
* Expose deterministic PRNG
* base32
* base58
* DL_Group
* EC_Group
Build/Test
----------------------------------------
* Support hardcoding all test vectors into the botan-test binary
so it can run as a standalone item (copied to a device, etc)
* Run iOS binary under simulator in CI
* Run Android binary under simulator in CI
* Run the TPM tests against an emulator
(https://github.com/PeterHuewe/tpm-emulator)
* Add support for vxWorks
CLI
----------------------------------------
* Add a ``--completion`` option to dump autocomplete info, write
support for autocompletion in bash/zsh.
* Refactor ``speed``
* Change `tls_server` to be a tty<->socket app, like `tls_client` is,
instead of a bogus echo server.
* `encrypt` / `decrypt` tools providing password based file encryption
* Add ECM factoring
* Clone of `minisign` signature utility
* Implementation of `tlsdate`
* Password store utility
* TOTP calculator
* Clone of magic wormhole
* ACVP client (https://github.com/usnistgov/ACVP)
Documentation
----------------------------------------
* Always needs help