androlyze.storage package

Submodules

androlyze.storage.Constants module

androlyze.storage.FileSysStorage module

class androlyze.storage.FileSysStorage.FileSysStorage(store_root_dir)[source]

Bases: object, androlyze.storage.ImportStorageInterface.ImportStorageInterface, androlyze.storage.ResultWritingInterface.ResultWritingInterface, androlyze.storage.apk.ApkCopyInterface.ApkCopyInterface

Manages the file system storage.

Attributes

Methods

APK_IMPORT_DIRNAME = 'apk'
APK_RES_DIRNAME = 'res'
copy_apk(apk, file_like_obj, **kwargs)[source]

Copy the apk to the file system (path specified through store_root_dir).

See also: ApkCopyInterface.copy_apk().

Parameters:

apk: Apk

Holds meta information needed to create the subdirectory names.

file_like_obj

A file-like object which holds the .apk data
Returns:

str

The path were the apk file has been copied
Raises:

IOError

FileSysCreateStorageStructureException
create_entry_for_apk(apk, update=False, tag=None)[source]

Create the file system path where the results will be kept for the specified apk file.

Parameters:

apk : Apk

update : bool, optional (default is False)

Has no effect here.

tag : str, optional (default is None)

Has no effect here.
Raises:

FileSysCreateStorageStructureException

If the directories could not be created or the apk file could not be opened.
create_filesys_structure(file_path)[source]

Create the file system path where the results will be kept for the specified file_path

Parameters:

file_path : str

The path structure to create.
Raises:

FileSysCreateStorageStructureException

If the directories could not be created or the apk file could not be opened.
del_store_root_dir()[source]
delete_entry_for_apk(apk, delete_apk=False)[source]

Delete the entry for apk.

Parameters:

apk: Apk

delete_apk : boolean, optional (default is False)

If true, also delete the .apk file from the file system (but only if it is in the storage directory!).
Raises:

FileSysDeleteException
fetch_results_from_mongodb(rds, results, wait_for_db=True, nice_progess=False, synced_entries=None, total_sync_entries=None)[source]

Fetch some results from the result database and write them to disk.

If data cannot be loaded from db, try until it can be.

Parameters:

rds : ResultDatabaseStorage

The database to query for the results.

results : list< tuple<id, gridfs (bool)> >

Define which results shall be fetched.

wait_for_db : bool, optional (default is True)

Wait until data could be fetched from db.

nice_progess : bool, optional (default is False)

If enabled update show some nice progress bar on the cli.

synced_entries : multiprocessing.Value<int>, optional (default is None)

If supplied store number of already synces entries.

total_sync_entries : multiprocessing.Value<int>, optional (default is None)

If supplied store number of total entries to sync.
Raises:

DatabaseLoadException

If wait_for_db is False and an error occurred.
get_apk_import_base_path()[source]

Returns the base path where the .apk will be stored

get_apk_import_file_name(apk)[source]

Returns the full path where the .apk will be stored.

Parameters:

apk : Apk

Returns

——

str
get_apk_import_path(apk)[source]

Returns the path structure for import directory of the apk.

Parameters:

apk: Apk
Returns:

str: path
Raises:

CouldNotOpenApk

If the APK could no be opened
get_apk_res_base_path()[source]

Returns the base path where the results of the apk analysis will be kept

get_apk_res_filename(apk, script)[source]

Return the path for the result filename.

Parameters:

apk : Apk

script : AndroScript
Raises:

CouldNotOpenApk

If the APK could no be opened
get_apk_res_path(apk)[source]

Returns the path structure for results of the apk analysis.

Parameters:

apk: Apk
Returns:

str: path
Raises:

CouldNotOpenApk

If the APK could no be opened
get_apk_res_path_all_args(package_name, version_name, _hash)[source]

Returns the path structure for result storage.

Parameters:

package_name : str

Package name of the apk. Unique apk identifier (at least in the store)

version_name : str

Version name

_hash : str

The hash of the apk.
Returns:

str: path
get_apk_sub_path(apk)[source]

Returns the sub path structure for the apk.

The structure is: ... |-> package

|-> version
|-> sha256
Parameters:

apk: Apk
Returns:

str: path
Raises:

CouldNotOpenApk

If the APK could no be opened
get_store_root_dir()[source]
is_in_store_dir(apk)[source]

Check if the path of the apk is in the store dir.

Parameters:apk : Apk
set_store_root_dir(value)[source]
store_custom_data(package_name, version_name, _hash, file_name, data)[source]

Store custom data to the file system (also with the result directory as root)

Parameters:

package_name : str

Package name of the apk. Unique apk identifier (at least in the store)

version_name : str

Version name

_hash : str

The hash of the apk.

file_name : str

File name.

data : object

The data what shall be written to disk. Will write str(data) to disk.
Raises:

FileSysStoreException
store_result_dict(res_dict)[source]

Store the analysis results from the res_dict. All needed infos for storage will be taken from it.

Parameters:

res_dict : dict

See ResultObject.description_dict
store_result_for_apk(apk, script)[source]

Store the results in the file system.

If a custom result object is used in script and it’s not a ResultObject, str(custom res object) will be used for writing to disk.

Parameters:

apk: Apk

script: AndroScript
Returns:

str

Path to result file.
Raises:

FileSysStoreException
store_root_dir

str - holds the path under which results will be stored

androlyze.storage.ImportStorageInterface module

class androlyze.storage.ImportStorageInterface.ImportStorageInterface[source]

Interface for the import storage

Methods

contains(apk)[source]

Check if the apk is present in the storage.

Parameters:apk: Apk
Returns:bool
create_entry_for_apk(apk, update=False, tag=None)[source]

Create an entry for the apk.

Will also update the path, if the file is already present in the database and has the same hash (at least if update).

Parameters:

apk : Apk

update : bool, optional (default is False)

Update an apk that has already been imported.

tag : str, optional (default is None)

Tag the apk with some text.
Raises:

StorageException
create_entry_for_apks(apks, update, tag=None)[source]

Create entry for the apks.

Parameters:

apk: iterable<Apk>

update : bool

Update apks that have already been imported.

tag : str, optional (default is None)

Tag the apk with some text.
delete_entry_for_apk(apk, delete_apk=False)[source]

Delete the entry for apk.

Parameters:

apk: Apk

delete_apk : boolean, optional (default is False)

If true, also delete the .apk file from the file system (but only if it is in the storage directory!).
Raises:

StorageException

androlyze.storage.RedundantStorage module

class androlyze.storage.RedundantStorage.RedundantStorage(import_db_name, store_root_dir=None, result_db_name=None, result_db_addr=None, result_db_port=None, result_db_username=None, result_db_passwd=None, result_db_use_ssl=False, ssl_ca_cert=None, distributed_apk_storage_factory=None)[source]

Bases: object, androlyze.storage.ImportStorageInterface.ImportStorageInterface, androlyze.storage.importdb.ImportQueryInterface.ImportQueryInterface, androlyze.storage.resultdb.ResultsStorageInterface.ResultStorageInterface, androlyze.storage.apk.ApkCopyInterface.ApkCopyInterface

The store is redundant in the sense that it stores the results in the file system as well as in the database. But the preferred choice is always database over file system. The file system is only a more convenient way to explore the results via command line.

The needed objects for the import db, file system and result db access will be created on demand!

Attributes

Methods

apk_distributed_storage

ApkCopyInterface : Distributed APK storage

contains(apk)[source]

Check if the apk has been imported yet. Query database due to performance.

Parameters:apk: Apk
Returns:bool
copy_apk(apk, file_like_obj, copy2fs=False, copy2db=False, **kwargs)[source]

See doc of ApkCopyInterface.copy_apk().

Returns both return values from ResultDatabaseStorage.copy_apk() as well as FileSysStorage.copy_apk().

Returns:list<str>
create_entry_for_apk(apk, update=False, tag=None, **kwargs)[source]

Create entry in file system as well as in database. Will set the import_date of the apk if not already imported. Also sets the tag.

Other Parameters:
 

no_db_import : bool, optional (default is False)

If true, don’t import into database. Just create file sys entry.

See documentation of `ImportStorageInterface.create_entry_for_apk`.
create_or_open_sub_storages()[source]

Create the FileSysStorage as well as the ResultDatabaseStorage. They are created and opened lazy on demand. This this method you force it to be created!

Raises:DatabaseOpenError
del_apk_distributed_storage()[source]
del_fs_storage()[source]
del_import_db_storage()[source]
del_result_db_storage()[source]
delete_entry_for_apk(apk, delete_apk=False)[source]

Delete the entry for apk.

Parameters:

apk: Apk

delete_apk : boolean, optional (default is False)

If true, also delete the .apk file from the file system (but only if it is in the storage directory!).
Raises:

StorageException
delete_results(*args, **kwargs)[source]

See doc of ResultDatabaseStorage.delete_results()

erase_whole_db()[source]

See doc of ResultDatabaseStorage.erase_whole_db()

fetch_results_from_mongodb(*args, **kwargs)[source]

See doc of FileSysStorage.fetch_results_from_mongodb()

fs_storage

FileSysStorage or nil: file system storage

fs_storage_disabled()[source]

Check if the result writing to disk is disabled

get_apk(_hash, *kwargs)[source]

See doc of ApkCopyInterface.copy_apk().

get_apk_distributed_storage()[source]
get_apk_hashes(package_names=None, tags=None)[source]
get_apk_package_names(hashes=None, tags=None)[source]
get_apk_paths(hashes=None, package_names=None, tags=None)[source]
get_fs_storage()[source]

Create FileSysStorage on demand.

If file sys result writing is disabled, return nil-like object that ignores all method calls and attribute lookups

Returns:

FileSysStorage

If result writing enabled.

nil

Otherwise.
get_import_db_storage()[source]

Create ImportDatabaseStorage on demand

get_imported_apks(hashes=None, package_names=None, tags=None, **kwargs)[source]
get_result_db_addr()[source]
get_result_db_ca_cert()[source]
get_result_db_name()[source]
get_result_db_port()[source]
get_result_db_storage()[source]

Create ResultDatabaseStorage on demand

get_result_db_use_ssl()[source]
get_results(*args, **kwargs)[source]

See doc of ResultDatabaseStorage.get_results()

get_results_for_ids(*args, **kwargs)[source]

See :py:method:`.ResultStorageInterface.get_results_for_ids`

get_versions(hashes=None, package_names=None, tags=None)[source]
import_db_storage

ImportDatabaseStorage: import database storage

result_db_addr

str : Address of mongodb database server.

result_db_ca_cert

str, optional (default is None) : The CA certificate

result_db_name

str,: The name of the database to use. Will be created if not already existing.

result_db_port

int : Port of mongodb database server.

result_db_storage

ResultDatabaseStorage : result database storage

result_db_use_ssl

bool, optional (default is False) : Use ssl for the connection.

set_apk_distributed_storage(value)[source]
set_fs_storage(value)[source]
set_import_db_storage(value)[source]
set_result_db_storage(value)[source]
store_result_for_apk(apk, script)[source]

Store the result for the apk in the file system as well as in the database.

If a custom result object is used in script and it’s not a ResultObject, str(custom res object) will be used for storage.

Does also some checks on the script!

Parameters:

apk: Apk

script: AndroScript
Returns:

See :py:method:`.ResultDatabaseStorage.store_result_for_apk`
Raises:

StorageException

androlyze.storage.ResultWritingInterface module

class androlyze.storage.ResultWritingInterface.ResultWritingInterface[source]

Interface for the writing of the analysis results.

Methods

static get_custom_res_obj_representation(script)[source]

Get the representation of the custom result object. This is the data repr. that shall stored

store_result_for_apk(apk, script)[source]

Store the result for the apk which has been analyzed with the script.

Will overwrite already existing results of the script in the storage

If a custom result object is used in script and it’s not a ResultObject, str(custom res object) will be used for writing to disk.

Parameters:

apk: Apk

script: AndroScript
Returns:

Dependent on the implementation
Raises:

StorageException

androlyze.storage.Util module

androlyze.storage.Util.escape_dict(_dict, escape_fct, escape_keys=True, escape_values=False)[source]

Escape the keys and/or values in the _dict with escape_fct.

Will do a deepcopy of the dict! So escaping isn’t in-place!

Parameters:

_dict : dict

escape_fct : object -> object

Escape function for key or value

escape_keys : bool, optional (default is True)

Apply escape_fct on keys

escape_values : bool, optional (default is False)

Apply escape_fct on values.
Returns:

dict
androlyze.storage.Util.get_apk_path(package_name, version_name, _hash)[source]

Returns the sub path structure by supplying all arguments needed for it.

The structure is: ... |-> package

|-> version
|-> sha256
Parameters:

package_name : str

Package name of the apk. Unique apk identifier (at least in the store)

version_name : str

Version name

_hash : str

The hash of the apk.
Returns:

str: path
androlyze.storage.Util.get_apk_path_incl_filename(apk)[source]

Returns the sub path structure by supplying all arguments needed for it.

The structure is: ... |-> package

|-> version
|-> sha256
Parameters:apk : Apk
Returns:str: path

Module contents