ibeis.init package

Submodules

ibeis.init._grave module

ibeis.init.filter_annots module

TODO:
  • cross validation
  • encounter vs database (time filtering)
ibeis.init.filter_annots.ensure_flatiterable(input_)[source]
ibeis.init.filter_annots.ensure_flatlistlike(input_)[source]
ibeis.init.filter_annots.expand_acfgs(ibs, aidcfg, verbose=None, use_cache=None, hack_exclude_keys=None, initial_aids=None, save_cache=True)[source]

Main multi-expansion function. Expands an annot config dict into qaids and daids. New version of this function based on a configuration dictionary built from command line argumetns

Parameters:
  • ibs (IBEISController) – ibeis controller object
  • aidcfg (dict) – configuration of the annotation filter
  • verbose (bool) – verbosity flag(default = False)
  • use_cache (bool) – turns on disk based caching(default = None)
  • hack_exclude_keys (None) – (default = None)
  • initial_aids (None) – (default = None)
Returns:

expanded_aids=(qaid_list, daid_list) - expanded list of aids that meet the criteria of the aidcfg filter

Return type:

tuple

TODO:

The database should be created first in most circumstances, then the queries should be filtered to meet the database restrictions? I’m not sure Sometimes you need to set the query aids constant, but sometimes you need to set the data aids constant. Seems to depend.

This function very much needs the idea of filter chains

OkNewIdea:
3 filters:
  • Common sampling - takes care of things like min time delta,
  • species, quality viewpoint etc.
  • query sampling
  • database sampling
Basic idea is
  • Sample large pool
  • Partition pool into query and database
Requires:
  • base sampling params
  • partition1 params
  • partition2 params
  • inter partition params?

CommandLine:

python -m ibeis.dev -e print_acfg  -a timectrl:qsize=10,dsize=10  --db PZ_MTEST --veryverbtd --nocache-aid
python -m ibeis.dev -e print_acfg  -a timectrl:qminqual=good,qsize=10,dsize=10  --db PZ_MTEST --veryverbtd --nocache-aid

python -m ibeis.dev -e print_acfg  -a timectrl --db PZ_MTEST --verbtd --nocache-aid
python -m ibeis.dev -e print_acfg  -a timectrl --db PZ_Master1 --verbtd --nocache-aid
python -m ibeis.dev -e print_acfg  -a timequalctrl --db PZ_Master1 --verbtd --nocache-aid

python -m ibeis.dev -e rank_cdf   -a controlled:qsize=10,dsize=10,dper_name=2 -t default --db PZ_MTEST
python -m ibeis.dev -e rank_cdf   -a controlled:qsize=10,dsize=20,dper_name=2 -t default --db PZ_MTEST
python -m ibeis.dev -e print      -a controlled:qsize=10,dsize=10             -t default --db PZ_MTEST --verbtd --nocache-aid

python -m ibeis.dev -e latexsum -t candinvar -a viewpoint_compare  --db NNP_Master3 --acfginfo
utprof.py -m ibeis.dev -e print -t candk -a varysize  --db PZ_MTEST --acfginfo
utprof.py -m ibeis.dev -e latexsum -t candk -a controlled  --db PZ_Master0 --acfginfo

python -m ibeis --tf get_annotcfg_list:0 --db NNP_Master3 -a viewpoint_compare --nocache-aid --verbtd

python -m ibeis --tf get_annotcfg_list  --db PZ_Master1 \
    -a timectrl:qhas_any=\(needswork,correctable,mildviewpoint\),qhas_none=\(viewpoint,photobomb,error:viewpoint,quality\) \
    --acfginfo --veryverbtd  --veryverbtd
python -m ibeis --tf draw_rank_cdf --db PZ_Master1 --show -t best \
    -a timectrl:qhas_any=\(needswork,correctable,mildviewpoint\),qhas_none=\(viewpoint,photobomb,error:viewpoint,quality\) \
    --acfginfo --veryverbtd

python -m ibeis --tf get_annotcfg_list  --db Oxford -a default:qhas_any=\(query,\),dpername=2,exclude_reference=True --acfginfo --verbtd  --veryverbtd --nocache-aid

CommandLine:

python -m ibeis.init.filter_annots --exec-expand_acfgs --show

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> import ibeis
>>> from ibeis.expt import annotation_configs
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> aidcfg = copy.deepcopy(annotation_configs.default)
>>> aidcfg['qcfg']['species'] = 'primary'
>>> initial_aids = None
>>> expanded_aids = expand_acfgs(ibs, aidcfg, initial_aids=initial_aids)
>>> result = ut.repr3(expanded_aids, nl=1, nobr=True)
>>> print(result)
[1, 2, 3, 4, 5, 6],
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
ibeis.init.filter_annots.expand_acfgs_consistently(ibs, acfg_combo, initial_aids=None, use_cache=None, verbose=None)[source]

Expands a set of configurations such that they are comparable

CommandLine:

python -m ibeis --tf parse_acfg_combo_list                  -a varysize
ibeis --tf get_annotcfg_list --db PZ_Master1 -a varysize
#ibeis --tf get_annotcfg_list --db lynx -a default:hack_imageset=True
ibeis --tf get_annotcfg_list --db PZ_Master1 -a varysize:qsize=None
ibeis --tf get_annotcfg_list --db PZ_Master0 --nofilter-dups  -a varysize
ibeis --tf get_annotcfg_list --db PZ_MTEST -a varysize --nofilter-dups
ibeis --tf get_annotcfg_list --db PZ_Master0 --verbtd                 --nofilter-dups -a varysize
ibeis --tf get_annotcfg_list --db PZ_Master1 -a viewpoint_compare                 --verbtd --nofilter-dups
ibeis --tf get_annotcfg_list -a timectrl --db GZ_Master1 --verbtd                 --nofilter-dups
ibeis.init.filter_annots.expand_single_acfg(ibs, aidcfg, verbose=None)[source]

for main_helpers

ibeis.init.filter_annots.expand_species(ibs, species, avail_aids=None)[source]
ibeis.init.filter_annots.filter_annots_general(ibs, aid_list=None, filter_kw={}, verbose=False, **kwargs)[source]
Parameters:
  • ibs (IBEISController) – ibeis controller object
  • aid_list (list) – list of annotation rowids
  • filter_kw
KWargs::
has_none_annotmatch, any_match_annotmatch, has_all, is_known, any_match_annot, logic_annot, none_match_annotmatch, max_num_annotmatch, any_startswith_annot, has_any, require_quality, species, any_match, view_ext, has_any_annotmatch, view_pername, max_num_annot, min_timedelta, any_startswith, max_numfeat, any_startswith_annotmatch, been_adjusted, any_endswith_annot, require_viewpoint, logic, has_any_annot, min_num_annotmatch, min_num, min_num_annot, has_all_annot, has_none, min_pername, any_endswith_annotmatch, any_endswith, require_timestamp, none_match, contrib_contains, has_all_annotmatch, logic_annotmatch, min_numfeat, none_match_annot, view_ext1, view_ext2, max_num, has_none_annot, minqual, view

CommandLine:

python -m ibeis --tf filter_annots_general
python -m ibeis --tf filter_annots_general --db PZ_Master1 \
        --has_any=[needswork,correctable,mildviewpoint] \
        --has_none=[viewpoint,photobomb,error:viewpoint,quality] --show

python -m ibeis --tf filter_annots_general --db=GZ_Master1  \
        --max-numfeat=300 --show --minqual=junk --species=None
python -m ibeis --tf filter_annots_general --db=lynx \
        --been_adjusted=True

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> import ibeis
>>> filter_kw = ut.argparse_dict(get_default_annot_filter_form(),
>>>                              type_hint=ut.ddict(list, has_any=list,
>>>                                                 has_none=list,
>>>                                                 logic=str))
>>> print('filter_kw = %s' % (ut.dict_str(filter_kw),))
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> aid_list = ibs.get_valid_aids()
>>> #filter_kw = dict(is_known=True, min_num=1, has_any='viewpoint')
>>> #filter_kw = dict(is_known=True, min_num=1, any_match='.*error.*')
>>> aid_list_ = filter_annots_general(ibs, aid_list, filter_kw)
>>> print('len(aid_list_) = %r' % (len(aid_list_),))
>>> all_tags = ut.flatten(ibs.get_annot_all_tags(aid_list_))
>>> filtered_tag_hist = ut.dict_hist(all_tags)
>>> ut.print_dict(filtered_tag_hist, key_order_metric='val')
>>> ut.print_dict(ibs.get_annot_stats_dict(aid_list_), 'annot_stats')
>>> ut.quit_if_noshow()
>>> import ibeis.viz.interact
>>> ibeis.viz.interact.interact_chip.interact_multichips(ibs, aid_list_)
>>> ut.show_if_requested()
ibeis.init.filter_annots.filter_annots_independent(ibs, avail_aids, aidcfg, prefix=u'', verbose=False, withpre=False)[source]

Filtering that doesn’t have to do with a reference set of aids

TODO make filterflags version

Parameters:
  • ibs (IBEISController) – ibeis controller object
  • avail_aids (list) –
  • aidcfg (dict) –
  • prefix (str) – (default = ‘’)
  • verbose (bool) – verbosity flag(default = False)
Returns:

avail_aids

Return type:

list

CommandLine:

python -m ibeis --tf filter_annots_independent --veryverbtd

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> import ibeis
>>> from ibeis.expt import annotation_configs
>>> ibs = ibeis.opendb(defaultdb='PZ_MTEST')
>>> avail_aids = input_aids = ibs.get_valid_aids()
>>> aidcfg = annotation_configs.default['dcfg']
>>> aidcfg['require_timestamp'] = True
>>> aidcfg['require_quality'] = False
>>> aidcfg['is_known'] = True
>>> prefix = ''
>>> verbose = True
>>> avail_aids = filter_annots_independent(ibs, avail_aids, aidcfg,
>>>                                        prefix, verbose)
>>> result = ('avail_aids = %s' % (str(avail_aids),))
>>> print(result)
ibeis.init.filter_annots.filter_annots_intragroup(ibs, avail_aids, aidcfg, prefix=u'', verbose=False, withpre=False)[source]

This filters annots using information about the relationships between the annotations in the avail_aids group. This function is not independent and a second consecutive call may yield new results. Thus, the order in which this filter is applied matters.

CommandLine:

ibeis --tf get_annotcfg_list \
        -a default:qsame_imageset=True,been_adjusted=True,excluderef=True \
        --db lynx --veryverbtd --nocache-aid

Example

>>> aidcfg['min_timedelta'] = 60 * 60 * 24
>>> aidcfg['min_pername'] = 3
ibeis.init.filter_annots.filterannots_by_tags(ibs, aid_list, filter_kw)[source]
Parameters:
  • ibs (IBEISController) – ibeis controller object
  • aid_list (list) – list of annotation rowids

CommandLine:

python -m ibeis --tf filterannots_by_tags
utprof.py -m ibeis --tf filterannots_by_tags
SeeAlso:
filter_annotmatch_by_tags

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='PZ_Master1')
>>> aid_list = ibs.get_valid_aids()
>>> has_any = ut.get_argval('--tags', type_=list,
>>>                         default=['SceneryMatch', 'Photobomb'])
>>> min_num = ut.get_argval('--min_num', type_=int, default=1)
>>> filter_kw = dict(has_any=has_any, min_num=1)
>>> aid_list_ = filterannots_by_tags(ibs, aid_list, filter_kw)
>>> print('aid_list_ = %r' % (aid_list_,))
>>> ut.quit_if_noshow()
>>> pass
>>> # TODO: show special annot group in GUI
ibeis.init.filter_annots.get_acfg_cacheinfo(ibs, aidcfg)[source]

Returns location and name of the ~~annot~~ data cache

ibeis.init.filter_annots.get_annot_tag_filterflags(ibs, aid_list, filter_kw, request_defaultkw=False)[source]

Filters annotations by tags including those that is belongs to in a pair

ibeis.init.filter_annots.get_default_annot_filter_form()[source]

Returns dictionary containing defaults for all valid filter parameters

CommandLine:

python -m ibeis --tf get_default_annot_filter_form

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> filter_kw = get_default_annot_filter_form()
>>> print(ut.dict_str(filter_kw, align=True))
>>> print(', '.join(filter_kw.keys()))
ibeis.init.filter_annots.get_reference_preference_order(ibs, gt_ref_grouped_aids, gt_avl_grouped_aids, prop_getter, cmp_func, aggfn, rng, verbose=False)[source]

Orders preference for sampling based on some metric

ibeis.init.filter_annots.hack_extra(ibs, expanded_aids)[source]
ibeis.init.filter_annots.hack_remove_label_errors(ibs, expanded_aids, verbose=None)[source]
ibeis.init.filter_annots.multi_sampled_seaturtle_queries()[source]
ibeis.init.filter_annots.sample_annots(ibs, avail_aids, aidcfg, prefix=u'', verbose=False)[source]

Sampling preserves input sample structure and thust does not always return exact values

CommandLine:

python -m ibeis --tf sample_annots --veryverbtd

python -m ibeis --tf get_annotcfg_list --db seaturtles             -a default:qhas_any=\(left,right\),sample_occur=True,exclude_reference=True,sample_offset=0,num_names=1 --acfginfo

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> import ibeis
>>> from ibeis.expt import annotation_configs
>>> ibs = ibeis.opendb(defaultdb='PZ_MTEST')
>>> avail_aids = input_aids = ibs.get_valid_aids()
>>> aidcfg = copy.deepcopy(annotation_configs.default['dcfg'])
>>> aidcfg['sample_per_name'] = 3
>>> aidcfg['sample_size'] = 10
>>> aidcfg['min_pername'] = 2
>>> prefix = ''
>>> verbose = True
>>> avail_aids = filter_annots_independent(ibs, avail_aids, aidcfg,
>>>                                        prefix, verbose)
>>> avail_aids = sample_annots(ibs, avail_aids, aidcfg,
>>>                            prefix, avail_aids)
>>> result = ('avail_aids = %s' % (str(avail_aids),))
>>> print(result)

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.init.filter_annots import *  # NOQA
>>> import ibeis
>>> from ibeis.expt import annotation_configs
>>> db = 'seaturtles'  # 'testdb1'
>>> ibs = ibeis.opendb(defaultdb=db)
>>> aidcfg = copy.deepcopy(annotation_configs.default)['qcfg']
>>> aidcfg['sample_occur'] = True
>>> initial_aids = ibs.get_valid_aids()
>>> withpre, verbose, prefix = True, 2, ''
>>> avail_aids = filter_annots_independent(
>>>     ibs, initial_aids, {'has_any': ['left', 'right']}, prefix, verbose)
>>> qaids = sample_annots(ibs, avail_aids, aidcfg, prefix, verbose)
>>> avail_aids = initial_aids
>>> ref_aids = qaids
>>> dcfg = dict(exclude_reference=True, sample_occur=True)
>>> daids = sample_annots_wrt_ref(ibs, initial_aids, dcfg, qaids, prefix, verbose)
>>> ibs.print_annotconfig_stats(qaids, daids, enc_per_name=True, per_enc=True)
ibeis.init.filter_annots.sample_annots_general(ibs, aid_list=None, filter_kw={}, verbose=False, **kwargs)[source]

filter + sampling

ibeis.init.filter_annots.sample_annots_wrt_ref(ibs, avail_aids, aidcfg, ref_aids, prefix=u'', verbose=False)[source]

Sampling when a reference set is given

ibeis.init.filter_annots.subindex_annots(ibs, avail_aids, aidcfg, ref_aids=None, prefix=u'', verbose=False)[source]

Returns exact subindex of annotations

ibeis.init.filter_annots.verb_context(filtertype, aidcfg, verbose)[source]

closure helper

ibeis.init.main_commands module

TODO: Rename to ibeis/init/commands.py

TODO; remove params module

ibeis.init.main_commands.postload_commands(ibs, back)[source]

Postload commands deal with a specific ibeis database

ibeis –db PZ_MTEST –occur “*All Images” –query 1 ibeis –db PZ_MTEST –occur “*All Images” –query-intra

ibeis.init.main_commands.preload_commands(dbdir, **kwargs)[source]

Preload commands work with command line arguments and global caches

ibeis.init.main_commands.preload_convert_hsdb(dbdir)[source]

Convert the database before loading (A bit hacky)

ibeis.init.main_commands.vdd(ibs)[source]

view data dir

ibeis.init.main_commands.vdq(dbdir)[source]

view directory and quit

ibeis.init.main_commands.vwd()[source]

view work dir

ibeis.init.main_helpers module

This module defines helper functions to access common input needed to test many functions. These functions give a rich command line interface to specifically select subsets of annotations, pipeline configurations, and other filters.

TODO: standardize function signatures

ibeis.init.main_helpers.testdata_aids(defaultdb=None, a=None, adefault=u'default', ibs=None, return_acfg=False, verbose=None, default_aids=None)[source]

Grabs default testdata for functions, but is command line overrideable

CommandLine:

python -m ibeis --tf testdata_aids --verbtd --db PZ_ViewPoints
python -m ibeis --tf testdata_aids --verbtd --db NNP_Master3 -a is_known=True,view_pername='#primary>0&#primary1>=1'
python -m ibeis --tf testdata_aids --verbtd --db PZ_Master1 -a default:is_known=True,view_pername='#primary>0&#primary1>=1'
python -m ibeis --tf testdata_aids --verbtd --db PZ_Master1 -a default:species=primary,minqual=ok --verbtd

python -m ibeis.other.dbinfo –test-latex_dbstats –dblist

CommandLine:

python -m ibeis.init.main_helpers --exec-testdata_aids --show

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> from ibeis.expt import annotation_configs
>>> import ibeis
>>> #ibs = ibeis.opendb(defaultdb='PZ_ViewPoints')
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> a = None
>>> adefault = 'default:is_known=True'
>>> aids, aidcfg = testdata_aids(ibs=ibs, a=a, adefault=adefault, return_acfg=True)
>>> print('\n RESULT:')
>>> annotation_configs.print_acfg(aidcfg, aids, ibs, per_name_vpedge=None)
ibeis.init.main_helpers.testdata_cm(defaultdb=None, default_qaids=None, t=None, p=None, a=None)[source]

CommandLine:

python -m ibeis.init.main_helpers --test-testdata_cm
python -m ibeis.init.main_helpers --test-testdata_cm --show

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> cm, qreq_ = testdata_cm()
>>> cm.print_csv(ibs=qreq_.ibs)
>>> ut.quit_if_noshow()
>>> cm.show_single_annotmatch(qreq_, 2)
>>> ut.show_if_requested()
ibeis.init.main_helpers.testdata_cmlist(defaultdb=None, default_qaids=None, t=None, p=None, a=None)[source]
Returns:cm_list, qreq_
Return type:list, ibeis.QueryRequest
ibeis.init.main_helpers.testdata_expanded_aids(defaultdb=None, a=None, ibs=None, default_qaids=None, default_daids=None, qaid_override=None, daid_override=None, return_annot_info=False, verbose=False, use_cache=None)[source]
Parameters:
  • default_qaids (list) – (default = [1])
  • default_daids (str) – (default = ‘all’)
  • defaultdb (str) – (default = ‘testdb1’)
  • ibs (IBEISController) – ibeis controller object(default = None)
  • verbose (bool) – verbosity flag(default = False)
  • return_annot_info (bool) – (default = False)
Returns:

Return type:

ibs, qaid_list, daid_list, annot_info

CommandLine:

python -m ibeis.init.main_helpers --exec-testdata_expanded_aids
python -m ibeis.init.main_helpers --exec-testdata_expanded_aids --db PZ_MTEST --acfg default:index=0:25 --verbose-testdata
python -m ibeis.init.main_helpers --exec-testdata_expanded_aids --db PZ_MTEST --qaid 3
python -m ibeis.init.main_helpers --exec-testdata_expanded_aids --db GZ_ALL --acfg ctrl --verbose-testdata

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> import ibeis
>>> from ibeis.expt import annotation_configs
>>> ibs, qaid_list, daid_list, aidcfg = testdata_expanded_aids(return_annot_info=True)
>>> print('Printing annot config')
>>> annotation_configs.print_acfg(aidcfg)
>>> print('Printing annotconfig stats')
>>> #print('qaid_list = %r' % (np.array(qaid_list),))
>>> ibs.print_annotconfig_stats(qaid_list, daid_list)
>>> print('Combined annotconfig stats')
>>> ibs.print_annot_stats(qaid_list + daid_list, yawtext_isect=True)
>>> print('qaid_list = %r' % (qaid_list,))
ibeis.init.main_helpers.testdata_expts(defaultdb=u'testdb1', default_acfgstr_name_list=[u'default:qindex=0:10:4,dindex=0:20'], default_test_cfg_name_list=[u'default'], a=None, t=None, p=None, qaid_override=None, daid_override=None, initial_aids=None, use_cache=None)[source]

Use this if you want data from an experiment. Command line interface to quickly get testdata for test_results.

Command line flags can be used to specify db, aidcfg, pipecfg, qaid override, daid override (and maybe initial aids).

CommandLine:

python -m ibeis.init.main_helpers testdata_expts

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.other.dbinfo import *  # NOQA
>>> import ibeis
>>> ibs, testres = ibeis.testdata_expts(defaultdb='PZ_MTEST', a='timectrl:qsize=2', t='invar:AI=[False],RI=False', use_cache=False)
>>> print('testres = %r' % (testres,))
ibeis.init.main_helpers.testdata_filtcfg(default=None)[source]
ibeis.init.main_helpers.testdata_pipecfg(p=None, t=None)[source]
Returns:pcfgdict
Return type:dict

CommandLine:

python -m ibeis.init.main_helpers --exec-testdata_pipecfg
python -m ibeis.init.main_helpers --exec-testdata_pipecfg -t default:AI=False

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> pcfgdict = testdata_pipecfg()
>>> result = ('pcfgdict = %s' % (ut.dict_str(pcfgdict),))
>>> print(result)
ibeis.init.main_helpers.testdata_qreq_(p=None, a=None, t=None, **kwargs)[source]
Parameters:t (None) – (default = None)
Kwargs:
default_qaids, a, defaultdb, ibs, verbose, return_annot_info
Returns:qreq_ - query request object with hyper-parameters
Return type:ibeis.QueryRequest

CommandLine:

python -m ibeis.init.main_helpers --exec-testdata_qreq_ --show --qaid 3

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> kwargs = {}
>>> p = None
>>> a = None
>>> qreq_ = testdata_qreq_(p)
>>> result = ('qreq_ = %s' % (str(qreq_),))

ibeis.init.old_main_helpers module

TODO: Rename to ibeis/init/commands.py

The AID configuration selection is getting a mjor update right now

ibeis.init.old_main_helpers.define_named_aid_cfgs()[source]

Definitions for common aid configurations TODO: potentially move to experiment configs

ibeis.init.old_main_helpers.ensure_flatiterable(input_)[source]
ibeis.init.old_main_helpers.ensure_flatlistlike(input_)[source]
ibeis.init.old_main_helpers.get_commandline_aidcfg()[source]

Parse the command line for “THE NEW AND IMPROVED” cannonical annotation configuration dictionaries

CommandLine:

python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg
python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg --qcfg default:shuffle=True,index=0:25 --dcfg default
python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg --qcfg default --dcfg default

python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg --qcfg controlled --dcfg controlled

python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg --acfg controlled

python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg --acfg varydbsize

python -m ibeis.init.main_helpers --exec-get_commandline_aidcfg --acfg controlled:qindex=0:10


--aidcfg=controlled=True,species=primary
--aidcfg=controlled=True,species=primary,annot_per_name=2
--aidcfg=controlled=True,species=primary,annot_per_name=3

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> aidcfg = get_commandline_aidcfg()
>>> print('aidcfg = ' + ut.dict_str(aidcfg))
ibeis.init.old_main_helpers.get_test_daids(ibs, default_daids='all', qaid_list=None, return_annot_info=False, aidcfg=None)[source]

Gets database annot_rowids based on command line arguments

DEPRICATE

CommandLine:

python dev.py --db PZ_MTEST -t best --exclude-query --qaid 72 -r 0 -c 0 --show --va --vf --dump-extra
Parameters:
  • ibs (IBEISController) – ibeis controller object
  • default_daids (str) – (default = ‘all’)
  • qaid_list (list) – list of chosen qaids that may affect daids (default = None)
Returns:

available_daids

Return type:

list

CommandLine:

python -m ibeis.init.main_helpers --test-get_test_daids
python -m ibeis.init.main_helpers --test-get_test_daids --db PZ_MTEST  --verbmhelp
python -m ibeis.init.main_helpers --test-get_test_daids --db PZ_MTEST --exclude-query
python -m ibeis.init.main_helpers --test-get_test_daids --db PZ_MTEST --daid-exclude 2 3 4
python -m ibeis.init.main_helpers --test-get_test_daids --db PZ_MTEST --species=zebra_grevys
python -m ibeis.init.main_helpers --test-get_test_daids --db PZ_Master0 --species=zebra_grevys
python -m ibeis.init.main_helpers --test-get_test_daids --db PZ_Master0 --controlled --verbmhelp
python -m ibeis.init.main_helpers --exec-get_test_daids --controlled --db PZ_Master0 --exec-mode

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> default_daids = 'all'
>>> qaid_list = [1]
>>> available_daids = get_test_daids(ibs, default_daids, qaid_list)
>>> ibeis.other.dbinfo.get_dbinfo(ibs, aid_list=available_daids, with_contrib=False, short=True)
>>> result = 'available_daids = ' + ut.obj_str(available_daids, truncate=True, nl=False)
>>> print('len(available_daids) %d' % len(available_daids))
>>> print(result)
available_daids = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
ibeis.init.old_main_helpers.get_test_qaids(ibs, default_qaids=None, return_annot_info=False, aidcfg=None)[source]

Gets test annot_rowids based on command line arguments

DEPRICATE

Parameters:
  • ibs (IBEISController) – ibeis controller object
  • default_qaids (None) – if list then used only if no other aids are available (default = [1]) as a string it mimics the command line
Returns:

available_qaids

Return type:

list

CommandLine:

python -m ibeis.init.main_helpers --test-get_test_qaids
python -m ibeis.init.main_helpers --test-get_test_qaids --controlled --db PZ_Master0
python -m ibeis.init.main_helpers --test-get_test_qaids --controlled --db PZ_Master0 --qaid 1
python -m ibeis.init.main_helpers --test-get_test_qaids --allgt --db PZ_MTEST
python -m ibeis.init.main_helpers --test-get_test_qaids --qaid 4 5 8  --verbmhelp
python -m ibeis.init.main_helpers --test-get_test_qaids --controlled --db PZ_MTEST
python -m ibeis.init.main_helpers --test-get_test_qaids --controlled --db PZ_MTEST --qaid 2 --verbmhelp
python -m ibeis.init.main_helpers --test-get_test_qaids --controlled --db PZ_MTEST --qaid 2
python -m ibeis.init.main_helpers --test-get_test_qaids --controlled --db PZ_Master0 --qindex 0:10 --verbmhelp
python -m ibeis.init.main_helpers --exec-get_test_qaids --controlled --db PZ_Master0 --exec-mode
python -m ibeis.init.main_helpers --exec-get_test_qaids --db testdb1 --allgt --qindex 0:256

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.main_helpers import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> default_qaids = None
>>> available_qaids = get_test_qaids(ibs, default_qaids)
>>> ibeis.other.dbinfo.get_dbinfo(ibs, aid_list=available_qaids, with_contrib=False, short=True)
>>> result = 'available_qaids = ' + ut.obj_str(available_qaids, truncate=True, nl=False)
>>> print('len(available_qaids) = %d' % len(available_qaids))
>>> print(result)
available_qaids = [1]

ibeis.init.sysres module

sysres.py == system_resources Module for dealing with system resoureces in the context of IBEIS but without the need for an actual IBEIS Controller

ibeis.init.sysres.copy_ibeisdb(source_dbdir, dest_dbdir)[source]
ibeis.init.sysres.db_to_dbdir(db, allow_newdir=False, extra_workdirs=[], use_sync=False)[source]

Implicitly gets dbdir. Searches for db inside of workdir

ibeis.init.sysres.ensure_db_from_url(zipped_db_url)[source]

SeeAlso ibeis.init.sysres

ibeis.init.sysres.ensure_nauts()[source]

Ensures that you have the NAUT_test dataset

ibeis.init.sysres.ensure_pz_mtest()[source]

Ensures that you have the PZ_MTEST dataset

CommandLine:

python -m ibeis.init.sysres --exec-ensure_pz_mtest
python -m ibeis --tf ensure_pz_mtest

Example

>>> # SCRIPT
>>> from ibeis.init.sysres import *  # NOQA
>>> ensure_pz_mtest()
ibeis.init.sysres.ensure_pz_mtest_batchworkflow_test()[source]

CommandLine:

python -m ibeis.init.sysres --test-ensure_pz_mtest_batchworkflow_test
python -m ibeis.init.sysres --test-ensure_pz_mtest_batchworkflow_test --reset
python -m ibeis.init.sysres --test-ensure_pz_mtest_batchworkflow_test --reset

Example

>>> # SCRIPT
>>> from ibeis.init.sysres import *  # NOQA
>>> ensure_pz_mtest_batchworkflow_test()
ibeis.init.sysres.ensure_pz_mtest_mergesplit_test()[source]

Make a test database for MERGE and SPLIT cases

CommandLine:

python -m ibeis.init.sysres --test-ensure_pz_mtest_mergesplit_test

Example

>>> # SCRIPT
>>> from ibeis.init.sysres import *  # NOQA
>>> ensure_pz_mtest_mergesplit_test()
ibeis.init.sysres.ensure_testdb2()[source]

SeeAlso ibeis.init.sysres

ibeis.init.sysres.ensure_wd_peter2()[source]

publish to lev:Leviathan:/media/hdd/PUBLIC/databases

CommandLine:

python -m ibeis.init.sysres --exec-ensure_wd_peter2

Example

>>> # SCRIPT
>>> from ibeis.init.sysres import *  # NOQA
>>> ensure_wd_peter2()
ibeis.init.sysres.ensure_wilddogs()[source]

Ensures that you have the NAUT_test dataset

ibeis.init.sysres.get_args_dbdir(defaultdb=None, allow_newdir=False, db=None, dbdir=None, cache_priority=False)[source]

Machinery for finding a database directory

such a hacky function with bad coding. Needs to just return a database dir and use the following priority dbdir, db, cache, something like that...

ibeis.init.sysres.get_available_databases(workdir=None)

Lists the available valid ibeis databases inside of a work directory

Parameters:workdir (None) –
Returns:ibsdb_list - ibeis controller object
Return type:IBEISController

CommandLine:

python -m ibeis.init.sysres --test-get_ibsdb_list

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.sysres import *  # NOQA
>>> workdir = None
>>> ibsdb_list = get_ibsdb_list(workdir)
>>> result = str('\n'.join(ibsdb_list))
>>> print(result)
ibeis.init.sysres.get_dbalias_dict()[source]
ibeis.init.sysres.get_default_dbdir()[source]
ibeis.init.sysres.get_global_distinctiveness_modeldir(ensure=True)[source]
ibeis.init.sysres.get_ibeis_resource_dir()[source]
ibeis.init.sysres.get_ibsdb_list(workdir=None)[source]

Lists the available valid ibeis databases inside of a work directory

Parameters:workdir (None) –
Returns:ibsdb_list - ibeis controller object
Return type:IBEISController

CommandLine:

python -m ibeis.init.sysres --test-get_ibsdb_list

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.sysres import *  # NOQA
>>> workdir = None
>>> ibsdb_list = get_ibsdb_list(workdir)
>>> result = str('\n'.join(ibsdb_list))
>>> print(result)
ibeis.init.sysres.get_logdir()[source]
ibeis.init.sysres.get_rawdir()[source]

Returns the standard raw data directory

ibeis.init.sysres.get_workdir(allow_gui=True)[source]

Returns the work directory set for this computer. If allow_gui is true, a dialog will ask a user to specify the workdir if it does not exist.

python -c “import ibeis; print(ibeis.get_workdir())”

Parameters:allow_gui (bool) – (default = True)
Returns:work_dir
Return type:str

CommandLine:

python -m ibeis.init.sysres --exec-get_workdir

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.sysres import *  # NOQA
>>> allow_gui = True
>>> work_dir = get_workdir(allow_gui)
>>> result = ('work_dir = %s' % (str(work_dir),))
>>> print(result)
ibeis.init.sysres.grab_example_smart_xml_fpath()[source]

Gets smart example xml

CommandLine:

python -m ibeis.init.sysres --test-grab_example_smart_xml_fpath

Example

>>> # DISABLE_DOCTEST
>>> import ibeis
>>> import os
>>> smart_xml_fpath = ibeis.sysres.grab_example_smart_xml_fpath()
>>> os.system('gvim ' + smart_xml_fpath)
>>> #ut.editfile(smart_xml_fpath)
ibeis.init.sysres.guiselect_workdir()[source]

Prompts the user to specify a work directory

ibeis.init.sysres.is_ibeisdb(path)[source]

Checks to see if path contains the IBEIS internal dir

ibeis.init.sysres.list_dbs(workdir=None)

Lists the available valid ibeis databases inside of a work directory

Parameters:workdir (None) –
Returns:ibsdb_list - ibeis controller object
Return type:IBEISController

CommandLine:

python -m ibeis.init.sysres --test-get_ibsdb_list

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.sysres import *  # NOQA
>>> workdir = None
>>> ibsdb_list = get_ibsdb_list(workdir)
>>> result = str('\n'.join(ibsdb_list))
>>> print(result)
ibeis.init.sysres.resolve_dbdir2(defaultdb=None, allow_newdir=False, db=None, dbdir=None)[source]

CommandLine:

python -m ibeis.init.sysres --exec-resolve_dbdir2 --db PZ_MTEST
python -m ibeis.init.sysres --exec-resolve_dbdir2 --db None
python -m ibeis.init.sysres --exec-resolve_dbdir2 --dbdir None
Parameters:
  • defaultdb (None) – (default = None)
  • allow_newdir (bool) – (default = False)
  • db (None) – (default = None)
  • dbdir (None) – (default = None)

CommandLine:

python -m ibeis.init.sysres --exec-resolve_dbdir2

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.init.sysres import *  # NOQA
>>> defaultdb = 'cache'
>>> allow_newdir = False
>>> dbdir_ = resolve_dbdir2(defaultdb)
>>> result = ('dbdir_ = %r' % (dbdir_,))
>>> print(result)
ibeis.init.sysres.set_default_dbdir(dbdir)[source]
ibeis.init.sysres.set_logdir(log_dir)[source]
ibeis.init.sysres.set_workdir(work_dir=None, allow_gui=True)[source]

Sets the workdirectory for this computer

Parameters:
  • work_dir (None) – (default = None)
  • allow_gui (bool) – (default = True)

CommandLine:

python -c "import ibeis; ibeis.sysres.set_workdir('/raid/work2')"
python -c "import ibeis; ibeis.sysres.set_workdir('/raid/work')"

python -m ibeis.init.sysres --exec-set_workdir --workdir

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.init.sysres import *  # NOQA
>>> print('current_work_dir = %s' % (str(get_workdir(False)),))
>>> work_dir = ut.get_argval('--workdir', type_=str, default=None)
>>> allow_gui = True
>>> result = set_workdir(work_dir, allow_gui)

Module contents