androlyze.storage.resultdb package

Submodules

androlyze.storage.resultdb.MongoUtil module

androlyze.storage.resultdb.MongoUtil.build_apk_meta_where(kwargs, gridfs=False)

Create where clause from kwargs for apk meta key

androlyze.storage.resultdb.MongoUtil.build_checks_filter(checks_non_empty_list=None, checks_empty_list=None, checks_true=None, checks_false=None, checks_not_null=None, checks_null=None, conjunction='or')

Helper function to easily check if some value has been set. E.g. == [],!= [], != null, == null, == true, == false.

Parameters:

checks_non_empty_list : iterable<str>, optional (default is ())

Check the keys against a non empty list.

checks_empty_list : iterable<str>, optional (default is ())

Check the keys against an empty list.

checks_true : iterable<str>, optional (default is ())

Check if the values of the given keys are true.

checks_false : iterable<str>, optional (default is ())

Check if the values of the given keys are false.

checks_not_null : iterable<str>, optional (default is ())

Check if the values of the given keys are null (python None).

checks_null : iterable<str>, optional (default is ())

Check if the values of the given keys are not null (python None).

conjunction : str, optional (default is ‘or’)

Choose between ‘or’ and ‘and’. Specifies how to to link the filter arguments.

Returns:

dict

Dictionary describing the checks. Can be used for mongodb.

Examples

>>> print build_checks_filter(checks_non_empty_list = ['logged.enum'], checks_true = ['logged.bool'])
{'$or': [{'logged.enum': {'$ne': []}}, {'logged.bool': True}]}
>>> print build_checks_filter(checks_empty_list = ["foo"])
{'foo': []}

androlyze.storage.resultdb.MongoUtil.build_script_meta_where(kwargs, gridfs=False)

Create where clause from kwargs for script meta key

androlyze.storage.resultdb.MongoUtil.escape_key(k)

Escape key k so that in conforms to mongodb’s key restrictions.

See also

http

//docs.mongodb.org/manual/faq/developers/#dollar-sign-operator-escaping

androlyze.storage.resultdb.MongoUtil.escape_keys(_dict)

Escape the keys in the _dict so that the _dict can be inserted into mongodb.

Will do a deepcopy of the dict! So escaping isn’t in-place!

Parameters:_dict : dict Returns:dict

androlyze.storage.resultdb.MongoUtil.format_query_result_db(res_cursor, distict_generator=False, count=False, raw=False, html=False)

Format the results from the result db (mongodb).

Parameters:

res_cursor : gridfs.grid_file.GridOutCursor or generator<object> or pymongo.cursor.Cursor

First if non_document and non_document_raw. Second if distinct values wanted. Thirst otherwise.

distict_generator : bool, optional (default is False)

Res is generator<object> created from the distinct(...) method of mongodb. If generaor<dict>, convert each dict to json. Otherwise just print.

count : bool, optional (default is False)

Only print count, not results

raw : bool, optional (default is False)

Print raw data from gridfs Otherwise print json. If raw will not be converted to html!

html : bool, optional (default is False)

Format as html.

Returns:

str

androlyze.storage.resultdb.MongoUtil.get_attr_str(key, attr, gridfs=False)

Get the attribute string depending on gridfs

androlyze.storage.resultdb.MongoUtil.is_pymongo_cursor(cursor)

Check if cursor is a mongodb cursor

androlyze.storage.resultdb.MongoUtil.split_result_ids(results)

Split the id’s into non-gridfs and gridfs id’s.

Parameters:

results : iterable<tuple<str, bool>>

First component is the id of the entry and the second a boolean indication if the result has been stored in gridfs. See e.g. output of :py:method:`.ResultDatabaseStorage.store_result_for_apk`

Returns:

tuple<list<str>, list<str>>

First component holds the non-gridfs id’s, the second the gridfs id’s

androlyze.storage.resultdb.ResultDatabaseStorage module

class androlyze.storage.resultdb.ResultDatabaseStorage.ResultDatabaseStorage(db_name=None, dest_addr=None, dest_port=None, username=None, passwd=None, use_ssl=False, ssl_ca_certs=None)

Bases: object, androlyze.storage.resultdb.ResultsStorageInterface.ResultStorageInterface, androlyze.storage.apk.ApkCopyInterface.ApkCopyInterface

Class for storing documents and/or binary data in mongodb.

Attributes

Methods

apk_coll

gridfs.GridFS : Apk collection (gridfs)

apk_db

pymongo.database.Database : Apk database

conn

pymongo.mongo_client.MongoClient : Mongodb connection

copy_apk(apk, file_like_obj, **kwargs)

See doc of ApkCopyInterface.copy_apk().

Inserts the apk from the file_like_obj into mongodb’s gridfs, but only if not already in db.

Returns:The id of the apk (in db)

create_where_clause(kwargs, from_gridfs=False)

Create where clause from kwargs.

Parameters:

from_gridfs : bool, optional (default is False)

Whether to build where clause for gridfs.

Other Parameters:  

package_name : str, optional (default is None)

apk_hash : str, optional (default is None)

version_name : str, optional (default is None)

tag : str, optional (default is None)

script_hash : str, optional (default is None)

script_name : str, optional (default is None)

script_version : str, optional (default is None)

Notes

If any of the other parameters is None it won’t be used for filtering.

db

pymongo.database.Database : Database

db_name

db_name : str, optional (default is ‘res’) - The name of the database to use. Will be created if not already existing.

del_apk_coll()

del_apk_db()

del_conn()

del_db()

del_db_name()

del_dest_addr()

del_dest_port()

del_files_coll()

del_grid_fs()

del_res_coll()

delete_results(where=None, non_document=False, **kwargs)

See doc of ResultStorageInterface.delete_results()

dest_addr

str, optional (default is ‘127.0.0.1’) : Address of mongodb database server.

dest_port

int, optional (default is 27017) : Port of mongodb database server.

erase_whole_db()

Use to drop collections and recreate them. See doc of ResultStorageInterface.erase_whole_db()

files_coll

pymongo.collection.Collection : files follection of gridfs

get_all_ids(where=None)

Get all id’s filtered by where.

Parameters:

where : dict, optional (default is None)

Dictionary doing the filtering. If not given, get all ids.

Returns

——

list<str>

get_apk(_hash, **kwargs)

Get the EAndroApk from _hash.

Parameters:

_hash : str

Hash of the .apk (sha256)

Returns:

EAndroApk

Apk constructed from raw data and meta infos.

Raises:

DatabaseLoadException

NoFile

If the file is not present.

get_apk_coll()

get_apk_db()

get_conn()

get_db()

get_db_name()

get_dest_addr()

get_dest_port()

get_eandro_apk(_id)

Get the EAndroApk from database.

Returns:

EAndroApk

None

If Apk could not be loaded

get_files_coll()

get_grid_fs()

get_ids(non_document=False, where=None)

Get the id’s for the results filtered by where.

Parameters:

non_document : bool, optional (default is False)

If True, use gridfs.

where : dict, optional (default is None)

Dictionary doing the filtering. If not given, get all ids.

Returns

——

list<str>

get_res_coll()

get_results(include_fields=None, exclude_fields=None, where=None, distinct_key=None, n=None, sort=True, latest=False, non_document=False, non_document_raw=False, remove_id_field=True, **kwargs)

See doc of ResultStorageInterface.get_results()

get_results_for_ids(ids, non_document=False, non_document_raw=False)

See :py:method:`.ResultStorageInterface.get_results_for_ids`

get_use_ssl()

grid_fs

gridfs.GridFS : Gridfs object for non-document and binary storage.

res_coll

pymongo.collection.Collection : results collection for documents

set_apk_coll(value)

set_apk_db(value)

set_conn(value)

set_db(value)

set_db_name(value)

set_dest_addr(value)

set_dest_port(value)

set_files_coll(value)

set_grid_fs(value)

set_res_coll(value)

store_result_for_apk(apk, script)

See doc of ResultWritingInterface.store_result_for_apk().

Returns:

tuple<str, bool>

First component is the id of the entry and the second a boolean indication if the result has been stored in gridfs.

None

If an error occurred.

use_ssl

bool, optional (default is False) : Use ssl for the connection.

androlyze.storage.resultdb.ResultDatabaseStorage.factory_from_config(settings)

Get a factory_from_config object from the distributed config.

Parameters:settings : Settings

androlyze.storage.resultdb.ResultsStorageInterface module

class androlyze.storage.resultdb.ResultsStorageInterface.ResultStorageInterface

Bases: androlyze.storage.ResultWritingInterface.ResultWritingInterface

Interface for the result storage

Methods

delete_results(where=None, non_document=False, **kwargs)

Delete some results from the database.

Parameters:

where : dict, optional (default is {})

A filter.

non_document : bool, optional (default is False)

Remove from gridfs.

Returns:

int

Number of documents which have been removed.

Other Parameters:  

package_name : str, optional (default is None)

apk_hash : str, optional (default is None)

version_name : str, optional (default is None)

tag : str, optional (default is None)

script_hash : str, optional (default is None)

script_name : str, optional (default is None)

script_version : str, optional (default is None)

Notes

If any of the other parameters is None it won’t be used for filtering. They may will also overwrite the other ones.

erase_whole_db()

Use to drop collections and recreate them.

get_results(include_fields=None, exclude_fields=None, where=None, distinct_key=None, n=None, sort=True, latest=False, non_document=False, non_document_raw=False, remove_id_field=True, **kwargs)

Get results from the database.

Parameters:

include_fields : list<str>, optional (default is [])

List of fields to include in the result. Mutually exclusive with exclude_fields.

exclude_fields : list<str>, optional (default is [])

List of fields to exclude from the result. Mutually exclusive with include_fields.

where : dict, optional (default is {})

A filter.

distinct_key : str, optional (default is None)

If given, list the distinct values for the `distinct_key.

sort : bool, optional (default is True)

If true sort by analysis date.

latest : bool, optional (default is False)

Get the result of the latest script run. Will only return one result.

n : int, optional (default is None)

Number of results to return. None means no limit.

non_document : bool, optional (default is False)

Get custom data from mongodb’s gridfs.

non_document_raw : bool, optional (default is False)

Get the raw data from the database. Otherwise meta infos will be returned. Only interesting if non_document.

remove_id_field : bool, optional (default is True)

Will remove the _id field by default.

Returns:

gridfs.grid_file.GridOutCursor

If non_document and non_document_raw.

pymongo.cursor.Cursor

Otherwise

generator<object>

If distinct_key

Other Parameters:  

package_name : str, optional (default is None)

apk_hash : str, optional (default is None)

version_name : str, optional (default is None)

tag : str, optional (default is None)

script_hash : str, optional (default is None)

script_name : str, optional (default is None)

script_version : str, optional (default is None)

Raises:

DatabaseLoadException

Notes

If any of the other parameters is None it won’t be used for filtering.

get_results_for_ids(ids, non_document=False, non_document_raw=False)

Get the results for the specified ids.

Parameters:

ids : iterable<str>

The ids to fetch the results for.

non_document : bool, optional (default is False)

If non_document fetch results from gridfs.

non_document_raw : bool, optional (default is False)

Get the raw data from the database. Otherwise meta infos will be returned. Only interesting if non_document. Means return GridOutCursor instead of fetching from metdata from files collection.

Returns:

gridfs.grid_file.GridOutCursor

If non_document.

pymongo.cursor.Cursor

Otherwise

Raises:

DatabaseLoadException

Module contents