Welcome to py_deps’s documentation!¶
Contents:
About py-dep¶
The py-dep
provides parsing the dependencies of Python packages
and generating the metadata for graph.
The graph data is for NetworkX, Linkdraw, etc.
Requirements¶
- Python 3.8
- setuptools 46.1.0 over
- pip 20.0. over
- NetworkX 2.4 over
- pylibmc 1.6.1 over (optional)
Features¶
- Generating Linkdraw data (JSON and decoded JSON).
- Generating Networkx DiGraph object data.
- Cache the parsed dependencies.
- Searching packages from PyPI.
Basic usage¶
py-deps provides parsing the Python deps and generating graph data.
Search packages¶
Search packages from PyPI.:
$ python
>>> from py_deps.deps import search
>>> search('deps')
[{'_pypi_ordering': False,
'name': 'anybox.recipe.sysdeps',
'summary': 'A buildout recipe to check system dependencies',
'version': '0.5'},
{'_pypi_ordering': False,
'name': 'appdynamics-bindeps-linux-x64',
'summary': 'Dependencies for AppDynamics Python agent',
'version': '4.0.5.0'},
{'_pypi_ordering': False,
'name': 'appdynamics-bindeps-linux-x86',
'summary': 'Dependencies for AppDynamics Python agent',
'version': '4.0.5.0'},
{'_pypi_ordering': False,
'name': 'appdynamics-bindeps-osx-x64',
'summary': 'Dependencies for AppDynamics Python agent',
'version': '4.0.5.0'},
{'_pypi_ordering': False,
'name': 'deps',
'summary': 'deps discovers your Python dependencies',
'version': '0.1.0'},
{'_pypi_ordering': False,
'name': 'gtkeggdeps',
'summary': 'Interactive egg dependency browser',
'version': '0.0.7'},
{'_pypi_ordering': False,
'name': 'htmldeps',
'summary': 'Expand CSS and javascript dependency links in HTML',
'version': '1.2.1'},
{'_pypi_ordering': False,
'name': 'py-deps',
'summary': 'parsing the Python deps and generating graph data',
'version': '0.3.0'},
{'_pypi_ordering': False,
'name': 'pydeps',
'summary': 'Display module dependencies',
'version': '1.2.1'},
{'_pypi_ordering': False,
'name': 'runestone-test-deps',
'summary': 'This is dependencies for RSI',
'version': '0.1'},
{'_pypi_ordering': False,
'name': 'tl.eggdeps',
'summary': 'Compute a dependency graph between active Python eggs.',
'version': '0.4'},
{'_pypi_ordering': False,
'name': 'tt.eggdeps',
'summary': 'Compute a dependency graph between active Python eggs.',
'version': '0.5'}]
Show latest version¶
Retrieve latest package from PyPI.:
$ python
>>> from py_deps.deps import latest_version
>>> latest_version('deps')
'0.1.0'
Initialize¶
Cache the parsed dependencies into the py-deps.pickle
on current working directory.This file format is
pickle.:
$ python
>>> from py_deps import Package
>>> pkg = Package('py-deps')
py-deps retrieve latest version from PyPI
without version
argument as above.
Specify version use version argument.:
>>> pkg = Package('py-deps', version='0.5.5')
>>> pkg.version
0.5.5
Change cache file¶
Use cache_name argument.:
>>> pkg = Package('py-deps', cache_name='some-cache.name')
Override cache forcely¶
Use update_force
argument. (default: False
):
>>> pkg = Package('py-deps', update_force=True)
Changes the cache backend to Memcached¶
Installing libmemcached-dev package and pylibmc.:
$ sudo apt-get install libmemcached-dev
$ . /path/to/venv/bin/activate
(venv)$ cd /path/to/py-deps
(venv)$ python setup.py memcache
Use servers
argument. The argment syntax follows pylibmc.Client.:
>>> pkg = Package('py-deps', servers=['127.0.0.1:11211'])
Generate rendering data¶
Supports follows currently.
- pretty print
- Linkdraw
Pretty print¶
>>> print(pkg.draw())
py-deps -> [networkx, pip, setuptools, wheel]
networkx -> [decorator]
>>>
Linkdraw¶
>>> import pprint
>>> pp = pprint.PrettyPrinter(width=120)
>>> pp.pprint(pkg.draw('linkdraw'))
{'descr': 'py-deps dependencies',
'lines': [{'color': '#d1e0fa', 'descr': '->', 'link': '', 'source': 'py-deps', 'target': 'networkx', 'width': '1'},
{'color': '#a4c1f4', 'descr': '->', 'link': '', 'source': 'networkx', 'target': 'decorator', 'width': '1'},
{'color': '#d1e0fa', 'descr': '->', 'link': '', 'source': 'py-deps', 'target': 'pip', 'width': '1'},
{'color': '#d1e0fa', 'descr': '->', 'link': '', 'source': 'py-deps', 'target': 'setuptools', 'width': '1'},
{'color': '#d1e0fa', 'descr': '->', 'link': '', 'source': 'py-deps', 'target': 'wheel', 'width': '1'}],
'nodes': [{'color': '#d1e0fa',
'depth': 0,
'link': 'https://github.com/mkouhei/py-deps',
'name': 'py-deps',
'r': '6',
'version': '0.5.5'},
{'color': '#a4c1f4',
'depth': 1,
'link': 'http://networkx.github.io/',
'name': 'networkx',
'r': '6',
'version': '2.4'},
{'color': '#76a1ef',
'depth': 2,
'link': 'https://github.com/micheles/decorator',
'name': 'decorator',
'r': '6',
'version': '4.4.2'},
{'color': '#a4c1f4', 'depth': 1, 'link': 'https://pip.pypa.io/', 'name': 'pip', 'r': '6', 'version': '20.1'},
{'color': '#a4c1f4',
'depth': 1,
'link': 'https://github.com/pypa/setuptools',
'name': 'setuptools',
'r': '6',
'version': '46.3.1'},
{'color': '#a4c1f4',
'depth': 1,
'link': 'https://github.com/pypa/wheel',
'name': 'wheel',
'r': '6',
'version': '0.34.2'}],
'time': '2020-05-16T14:06:52.790139'}
See also How to use linkdraw.
NetworkX¶
>>> pkg.draw('networkx')
>>> <networkx.classes.digraph.DiGraph at 0x7fbe2311dbd0>
Check cache¶
Stores parsed dependency metadata to pickles data file.
The file name is py-deps.pickle
in default.
Listing cached data with the list_data
method of Container
.:
>>> from py_deps.cache import backend
>>> backend().list_data()
{('py-deps', '0.5.5'): [py-deps]}
(snip)}
Read the cached package with read_data
method of Container
.
This method returns Package.traced_chain
.
>>> Container().read_data(('py-deps', '0.5.5'))
[py-deps]
References¶
py_deps.deps module.
-
py_deps.deps.
SUFFIX
= '-py_deps'¶ suffix of temporary directory name
-
py_deps.deps.
u2h
(name)[source]¶ Change underscore to hyphen of package name.
Return type: str Returns: string replaced underscore with hyphen Parameters: name (str) – package name
-
py_deps.deps.
search
(pkg_name, exactly=False)[source]¶ Search package.
Return type: list
Returns: search packages
Parameters: - pkg_name (str) – package name.
- exactly (bool) – exactly match only.
-
py_deps.deps.
latest_version
(pkg_name)[source]¶ Retrieve latest version.
Return type: str Returns: latest version Parameters: pkg_name (str) – package name.
-
py_deps.deps.
create_nodes
(package_names, depth=0)[source]¶ Show information about installed package.
-
class
py_deps.deps.
Package
(name, version=None, update_force=False, **kwargs)[source]¶ Bases:
object
Package class.
-
index_url
= 'https://pypi.python.org/simple'¶ index_url
-
name
= None¶ package name
-
-
class
py_deps.deps.
Node
(name, version=None, url=None, requires=None, depth=0)[source]¶ Bases:
object
Node object class.
-
name
= None¶ name
-
version
= None¶ version
-
url
= None¶ project url
-
requires
= None¶ requires
-
targets
= None¶ targets
-
test_targets
= None¶ test targets
-
depth
= None¶ base dependency depth level
-
py_deps.graph module.
-
py_deps.graph.
pretty_print
(chain_data)[source]¶ Pretty print on terminal.
Parameters: chain_data (list) – List of deps.Node
-
class
py_deps.graph.
Graph
(package, link_prefix=None)[source]¶ Bases:
object
Graph data generate abstract class.
-
class
py_deps.graph.
Linkdraw
(package, link_prefix=None)[source]¶ Bases:
py_deps.graph.Graph
Linkdraw object class.
-
class
py_deps.graph.
Networkx
(package, link_prefix=None)[source]¶ Bases:
py_deps.graph.Graph
Networkx object class.
-
py_deps.graph.
color
(depth)[source]¶ Color by depth level.
Return type: str Returns: hex color code based blue. Parameters: depth (int) – dependency level
py_deps.cache module.
-
py_deps.cache.
backend
(**kwargs)[source]¶ Specify cache backend.
Return type: py_deps.cache.Container
Returns: Pickle object or Memcached object. Parameters: kwargs – parameters - servers
Memcached servers (required in Memcached)
Required when Memcached. Using Pickle in default without
servers
.- username
- Memcached SASL username (optional)
- password
- Memcached SASL password (optional)
- cache_name
- Pickle filename (default, optional)
-
class
py_deps.cache.
Container
(cache_name=None)[source]¶ Bases:
object
Package container class.
-
class
py_deps.cache.
Pickle
(cache_name=None)[source]¶ Bases:
py_deps.cache.Container
Cache backend is Pickle.
-
default_cache_name
= 'py-deps.pickle'¶ default cache file name
-
store_data
(key, data)[source]¶ Store traced_chain data.
Parameters: - key (tuple) – package name, version
- data (list) – traced dependency chain data
-
list_data
()¶ Return dictionary stored package metadata.
Return type: dict Returns: packages metadata
-
read_data
(key)¶ Read traced_chain data.
Return type: list Returns: dependency chain list Parameters: key (tuple) – package name, version
-
-
class
py_deps.cache.
Memcached
(servers=None, username=None, password=None, behaviors=None)[source]¶ Bases:
py_deps.cache.Container
Cache backend is Memecached.
-
store_data
(key, data)[source]¶ Store traced_chain data.
Parameters: - key (tuple) – package name, version
- data (list) – traced dependency chain data
-
list_data
()¶ Return dictionary stored package metadata.
Return type: dict Returns: packages metadata
-
py_deps.exceptions module.
-
exception
py_deps.exceptions.
Error
[source]¶ Bases:
Exception
Base error class.
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
exception
py_deps.exceptions.
NotFound
[source]¶ Bases:
py_deps.exceptions.Error
Not Found.
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
exception
py_deps.exceptions.
BrokenPackage
[source]¶ Bases:
py_deps.exceptions.Error
BrokenPackage.
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
exception
py_deps.exceptions.
InvalidMetadata
[source]¶ Bases:
py_deps.exceptions.Error
Invalid package metadata.
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
-
exception
py_deps.exceptions.
BackendFailure
[source]¶ Bases:
py_deps.exceptions.Error
PyPI service down.
-
with_traceback
()¶ Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
-
History¶
1.0.1 (2020-09-19)¶
- Fixes raise-missing-from, super-with-arguments pylint violations.
1.0.0 (2020-05-16)¶
- Updates dependencies.
- Refactors.
- Supports Python 3.8 only.
0.5.5 (2015-08-19)¶
- Adds TimeoutError, ConnectionRefusedError / socket.error exceptions.
- Adds error handling the PyPI service down.
- Changes Sphinx theme to sphinx_rtd_theme.
- Adds counter of the each depth.
0.5.4 (2015-07-22)¶
- Adds latest_version function.
- Adds link_prefix for overriding node link.
0.5.3 (2015-07-19)¶
- Adds exception InvalidMetadata type.
0.5.2 (2015-07-15)¶
- Fixes duplicated line of linkdraw.
- Changes linkdraw colors by dependencies depth.
- Adds depth property to graph nodes.
- Adds parsing the package dependency depth.
0.5.1 (2015-07-12)¶
- Changes Package.search method to a function.
- Fixes infinity loop trace_chain.
- Fixes None redundant second argument of dict.get().
- Fixes len() used to check if collection has items.
- Fixes old-style string formatting.
0.5.0 (2015-06-22)¶
- Supports memcached as the backend of cache.
0.4.6 (2015-06-11)¶
- Fixes not control the version of package correctly.
0.4.5 (2015-06-07)¶
- Adds disable time, descr for Linkdraw.
0.4.4 (2015-06-03)¶
- Removes debug print.
0.4.3 (2015-06-02)¶
- Adds JSON decoder for Linkdraw.
0.4.2 (2015-05-31)¶
- Fixes #7 not handling the failure of python setup egg_info.
- Adds py_deps.exceptions module.
- Adds py_deps.logger module.
- Fixes issues of DistributionNotFound, InstallationErrror.
0.4.1 (2015-05-28)¶
- Adds Container.list_data method.
- Unsupports wheel format for distribution.
0.4.0 (2015-05-20)¶
- Searching packages from PyPI.
0.3.0 (2015-05-12)¶
- Supports NetworkX DiGraph objects.
- Changes to use mock instead of pip.req.RequirementSet.prepare_files.
- Coverage 98% over.
0.2.0 (2015-05-10)¶
- Cache the parsed dependencies.
- Fixes setting the url of node and targets.
0.1.1 (2015-05-08)¶
- Fixes test data of pretty_print, linkdraw.
0.1.0 (2015-05-07)¶
- Supports generating linkdraw data.
- Supports pip 6.1.1 over.
- Supports wheel format for distribution.
- Adds unit tests.
0.0.1 (2015-04-29)¶
- First release