ibeis.dbio package¶

Submodules¶

ibeis.dbio.export_hsdb module¶

Converts an IBEIS database to a hotspotter db

ibeis.dbio.export_hsdb.dump_hots_flat_table(ibs)[source]¶

ibeis.dbio.export_hsdb.dump_hots_tables(ibs)[source]¶

Dumps hotspotter like tables to disk

ibeis.dbio.export_hsdb.export_ibeis_to_hotspotter(ibs)[source]¶

ibeis.dbio.export_hsdb.get_hots_flat_table(ibs)[source]¶

Dumps hotspotter flat tables

Parameters:	ibs (IBEISController) – ibeis controller object
Returns:	flat_table_str
Return type:	str

CommandLine:

python -m ibeis.dbio.export_hsdb --exec-get_hots_flat_table

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.dbio.export_hsdb import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> flat_table_str = get_hots_flat_table(ibs)
>>> result = ('flat_table_str = %s' % (str(flat_table_str),))
>>> print(result)

ibeis.dbio.export_hsdb.get_hots_table_strings(ibs)[source]¶

Parameters:	ibs (IBEISController) – ibeis controller object

CommandLine:

python -m ibeis.dbio.export_hsdb --test-get_hots_table_strings

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.dbio.export_hsdb import *  # NOQA
>>> import ibeis
>>> # build test data
>>> ibs = ibeis.opendb('testdb1')
>>> ibs.delete_empty_nids()
>>> # execute function
>>> csvtup = get_hots_table_strings(ibs)
>>> # hack so hashtag is at the end of each line
>>> result = '\n'.join(csvtup).replace('\n', '#\n') + '#'
>>> # verify results
>>> print(result)
# image table#
# num_rows=13#
#   gid,                                                      gname,  aif#
      1,  ../_ibsdb/images/66ec193a-1619-b3b6-216d-1784b4833b61.jpg,    0#
      2,  ../_ibsdb/images/d8903434-942f-e0f5-d6c2-0dcbe3137bf7.jpg,    0#
      3,  ../_ibsdb/images/b73b72f4-4acb-c445-e72c-05ce02719d3d.jpg,    0#
      4,  ../_ibsdb/images/0cd05978-3d83-b2ee-2ac9-798dd571c3b3.jpg,    0#
      5,  ../_ibsdb/images/0a9bc03d-a75e-8d14-0153-e2949502aba7.jpg,    0#
      6,  ../_ibsdb/images/2deeff06-5546-c752-15dc-2bd0fdb1198a.jpg,    0#
      7,  ../_ibsdb/images/a9b70278-a936-c1dd-8a3b-bc1e9a998bf0.png,    0#
      8,  ../_ibsdb/images/42fdad98-369a-2cbc-67b1-983d6d6a3a60.jpg,    0#
      9,  ../_ibsdb/images/c459d381-fd74-1d99-6215-e42e3f432ea9.jpg,    0#
     10,  ../_ibsdb/images/33fd9813-3a2b-774b-3fcc-4360d1ae151b.jpg,    0#
     11,  ../_ibsdb/images/97e8ea74-873f-2092-b372-f928a7be30fa.jpg,    0#
     12,  ../_ibsdb/images/588bc218-83a5-d400-21aa-d499832632b0.jpg,    0#
     13,  ../_ibsdb/images/163a890c-36f2-981e-3529-c552b6d668a3.jpg,    0#
# name table#
# num_rows=7#
#   nid,   name#
      1,   easy#
      2,   hard#
      3,   jeff#
      4,   lena#
      5,   occl#
      6,  polar#
      7,  zebra#
# chip table#
# num_rows=13#
#   cid,  gid,  nid,      [tlx tly w h],  theta,                                        notes#
      1,    1,   -1,  [0  0  1047  715],   0.00,              aid 1 and 2 are correct matches#
      2,    2,    1,  [0  0  1035  576],   0.00,                                             #
      3,    3,    1,  [0  0  1072  804],   0.00,                                             #
      4,    4,   -4,  [0  0  1072  804],   0.00,                                             #
      5,    5,    2,  [0  0  1072  804],   0.00,                                             #
      6,    6,    2,   [0  0  450  301],   0.00,                                             #
      7,    7,    3,   [0  0  400  400],   0.00,  very simple image to debug feature detector#
      8,    8,    4,   [0  0  220  220],   0.00,                          standard test image#
      9,    9,   -9,   [0  0  450  284],   0.00,              this is actually a plains zebra#
     10,   10,    5,   [0  0  450  341],   0.00,              this is actually a plains zebra#
     11,   11,  -11,   [0  0  741  734],   0.00,                                             #
     12,   12,    6,   [0  0  673  634],   0.00,                                             #
     13,   13,    7,  [0  0  1114  545],   0.00,                                             #

ibeis.dbio.export_hsdb.get_hsdb_image_gpaths(ibs, gid_list)[source]¶

Parameters:	ibs (IBEISController) – ibeis controller object gid_list (list) –
Returns:	gpath_list
Return type:	list

CommandLine:

python -m ibeis.dbio.export_hsdb --test-get_hsdb_image_gpaths

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.dbio.export_hsdb import *  # NOQA
>>> import ibeis
>>> # build test data
>>> ibs = ibeis.opendb('testdb1')
>>> gid_list = ibs.get_valid_gids()[0:2]
>>> # execute function
>>> gpath_list = get_hsdb_image_gpaths(ibs, gid_list)
>>> # verify results
>>> result = ut.list_str(gpath_list)
>>> print(result)
[
    '../_ibsdb/images/66ec193a-1619-b3b6-216d-1784b4833b61.jpg',
    '../_ibsdb/images/d8903434-942f-e0f5-d6c2-0dcbe3137bf7.jpg',
]

ibeis.dbio.export_subset module¶

Exports subset of an IBEIS database to a new IBEIS database

class ibeis.dbio.export_subset.ANNOTATION_TransferData(annot_parent_INDEX_list, annot_uuid_list, annot_theta_list, annot_verts_list, annot_yaw_list, annot_detection_confidence_list, annot_exemplar_flag_list, annot_visual_uuid_list, annot_semantic_uuid_list, annot_note_list, annot_name_INDEX_list, annot_species_INDEX_list, lblannot_td_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

annot_detection_confidence_list¶

Alias for field number 5

annot_exemplar_flag_list¶

Alias for field number 6

annot_name_INDEX_list¶

Alias for field number 10

annot_note_list¶

Alias for field number 9

annot_parent_INDEX_list¶

Alias for field number 0

annot_semantic_uuid_list¶

Alias for field number 8

annot_species_INDEX_list¶

Alias for field number 11

annot_theta_list¶

Alias for field number 2

annot_uuid_list¶

Alias for field number 1

annot_verts_list¶

Alias for field number 3

annot_visual_uuid_list¶

Alias for field number 7

annot_yaw_list¶

Alias for field number 4

lblannot_td_list¶

Alias for field number 12

class ibeis.dbio.export_subset.CONFIG_TransferData(config_suffixes_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

config_suffixes_list¶

Alias for field number 0

class ibeis.dbio.export_subset.CONTRIBUTOR_TransferData(contributor_uuid, contributor_tag, contributor_name_first, contributor_name_last, contributor_location_city, contributor_location_state, contributor_location_country, contributor_location_zip, contributor_note, config_td, imageset_td, image_td)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

config_td¶

Alias for field number 9

contributor_location_city¶

Alias for field number 4

contributor_location_country¶

Alias for field number 6

contributor_location_state¶

Alias for field number 5

contributor_location_zip¶

Alias for field number 7

contributor_name_first¶

Alias for field number 2

contributor_name_last¶

Alias for field number 3

contributor_note¶

Alias for field number 8

contributor_tag¶

Alias for field number 1

contributor_uuid¶

Alias for field number 0

image_td¶

Alias for field number 11

imageset_td¶

Alias for field number 10

class ibeis.dbio.export_subset.IMAGESET_TransferData(config_INDEX_list, imageset_uuid_list, imageset_text_list, encoutner_note_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

config_INDEX_list¶

Alias for field number 0

encoutner_note_list¶

Alias for field number 3

imageset_text_list¶

Alias for field number 2

imageset_uuid_list¶

Alias for field number 1

class ibeis.dbio.export_subset.IMAGE_TransferData(imageset_INDEXs_list, image_path_list, image_uuid_list, image_ext_list, image_original_name_list, image_width_list, image_height_list, image_time_posix_list, image_gps_lat_list, image_gps_lon_list, image_toggle_enabled_list, image_toggle_reviewed_list, image_note_list, lblimage_td_list, annotation_td_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

annotation_td_list¶

Alias for field number 14

image_ext_list¶

Alias for field number 3

image_gps_lat_list¶

Alias for field number 8

image_gps_lon_list¶

Alias for field number 9

image_height_list¶

Alias for field number 6

image_note_list¶

Alias for field number 12

image_original_name_list¶

Alias for field number 4

image_path_list¶

Alias for field number 1

image_time_posix_list¶

Alias for field number 7

image_toggle_enabled_list¶

Alias for field number 10

image_toggle_reviewed_list¶

Alias for field number 11

image_uuid_list¶

Alias for field number 2

image_width_list¶

Alias for field number 5

imageset_INDEXs_list¶

Alias for field number 0

lblimage_td_list¶

Alias for field number 13

class ibeis.dbio.export_subset.LBLANNOT_TransferData(config_INDEX_list, alr_confidence_list, lblannot_uuid_list, lbltype_text_list, lblannot_value_list, lblannot_note_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

alr_confidence_list¶

Alias for field number 1

config_INDEX_list¶

Alias for field number 0

lblannot_note_list¶

Alias for field number 5

lblannot_uuid_list¶

Alias for field number 2

lblannot_value_list¶

Alias for field number 4

lbltype_text_list¶

Alias for field number 3

class ibeis.dbio.export_subset.LBLIMAGE_TransferData(config_INDEX_list, glr_confidence_list, lblimage_uuid_list, lbltype_text_list, lblimage_value_list, lblimage_note_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

config_INDEX_list¶

Alias for field number 0

glr_confidence_list¶

Alias for field number 1

lblimage_note_list¶

Alias for field number 5

lblimage_uuid_list¶

Alias for field number 2

lblimage_value_list¶

Alias for field number 4

lbltype_text_list¶

Alias for field number 3

class ibeis.dbio.export_subset.NAME_TransferData(name_uuid_list, name_text_list, name_note_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

name_note_list¶

Alias for field number 2

name_text_list¶

Alias for field number 1

name_uuid_list¶

Alias for field number 0

class ibeis.dbio.export_subset.SPECIES_TransferData(species_uuid_list, species_nice_list, species_text_list, species_code_list, species_note_list)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

species_code_list¶

Alias for field number 3

species_nice_list¶

Alias for field number 1

species_note_list¶

Alias for field number 4

species_text_list¶

Alias for field number 2

species_uuid_list¶

Alias for field number 0

class ibeis.dbio.export_subset.TransferData(transfer_database_name, transfer_database_source, transfer_export_time, transfer_export_location_city, transfer_export_location_state, transfer_export_location_zip, transfer_export_location_country, contributor_td_list, name_td, species_td)¶

Bases: tuple

__getnewargs__()¶

Return self as a plain tuple. Used by copy and pickle.

__getstate__()¶

Exclude the OrderedDict from pickling

__repr__()¶

Return a nicely formatted representation string

contributor_td_list¶

Alias for field number 7

name_td¶

Alias for field number 8

species_td¶

Alias for field number 9

transfer_database_name¶

Alias for field number 0

transfer_database_source¶

Alias for field number 1

transfer_export_location_city¶

Alias for field number 3

transfer_export_location_country¶

Alias for field number 6

transfer_export_location_state¶

Alias for field number 4

transfer_export_location_zip¶

Alias for field number 5

transfer_export_time¶

Alias for field number 2

ibeis.dbio.export_subset.check_database_overlap(ibs1, ibs2)[source]¶

CommandLine:

python -m ibeis.other.dbinfo --test-get_dbinfo:1 --db PZ_MTEST
dev.py -t listdbs
python -m ibeis.dbio.export_subset --exec-check_database_overlap
--db PZ_MTEST --db2 PZ_MOTHERS

CommandLine:

python -m ibeis.dbio.export_subset --exec-check_database_overlap

python -m ibeis.dbio.export_subset --exec-check_database_overlap --db1=PZ_MTEST --db2=PZ_Master0  # NOQA
python -m ibeis.dbio.export_subset --exec-check_database_overlap --db1=NNP_Master3 --db2=PZ_Master0  # NOQA

python -m ibeis.dbio.export_subset --exec-check_database_overlap --db1=GZ_Master0 --db2=GZ_ALL
python -m ibeis.dbio.export_subset --exec-check_database_overlap --db1=GZ_ALL --db2=lewa_grevys

python -m ibeis.dbio.export_subset --exec-check_database_overlap --db1=PZ_FlankHack --db2=PZ_Master1

Example

>>> # SCRIPT
>>> import ibeis
>>> #ibs1 = ibeis.opendb(db='PZ_Master0')
>>> #ibs2 = ibeis.opendb(dbdir='/raid/work2/Turk/PZ_Master')
>>> db1 = ut.get_argval('--db1', str, default='PZ_MTEST')
>>> db2 = ut.get_argval('--db2', str, default='testdb1')
>>> dbdir1 = ut.get_argval('--dbdir1', str, default=None)
>>> dbdir2 = ut.get_argval('--dbdir2', str, default=None)
>>> ibs1 = ibeis.opendb(db=db1, dbdir=dbdir1)
>>> ibs2 = ibeis.opendb(db=db2, dbdir=dbdir2)
>>> check_database_overlap(ibs1, ibs2)

ibeis.dbio.export_subset.check_merge(ibs_src, ibs_dst)[source]¶

ibeis.dbio.export_subset.export_annot_transfer_data(ibs_src, aid_list, config_rowid_list, nid_list, species_rowid_list)[source]¶

ibeis.dbio.export_subset.export_annots(ibs, aid_list, new_dbpath=None)[source]¶

exports a subset of annotations and other required info

TODO:

PZ_Master1 needs to backproject information back on to NNP_Master3 and PZ_Master0

Parameters:	ibs (IBEISController) – ibeis controller object aid_list (list) – list of annotation rowids new_dbpath (None) – (default = None)
Returns:	new_dbpath
Return type:	str

CommandLine:

python -m ibeis.dbio.export_subset --exec-export_annots
python -m ibeis.expt.experiment_helpers --exec-get_annotcfg_list:0 --db NNP_Master3 -a viewpoint_compare --nocache-aid --verbtd
python -m ibeis.expt.experiment_helpers --exec-get_annotcfg_list:0 --db NNP_Master3 -a viewpoint_compare --nocache-aid --verbtd

python -m ibeis.dbio.export_subset --exec-export_annots --db NNP_Master3 -a viewpoint_compare --nocache-aid --verbtd --new_dbpath=PZ_ViewPoints

python -m ibeis.expt.experiment_helpers --exec-get_annotcfg_list:0 --db NNP_Master3 -a default:aids=all,is_known=True,view_pername=#primary>0&#primary1>0,per_name=4,size=200
python -m ibeis.expt.experiment_helpers --exec-get_annotcfg_list:0 --db NNP_Master3 -a default:aids=all,is_known=True,view_pername='#primary>0&#primary1>0',per_name=4,size=200 --acfginfo

Example

>>> # SCRIPT
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> import ibeis
>>> from ibeis.expt import experiment_helpers
>>> ibs = ibeis.opendb(defaultdb='NNP_Master3')
>>> acfg_name_list = ut.get_argval(('--aidcfg', '--acfg', '-a'), type_=list, default=[''])
>>> acfg_list, expanded_aids_list = experiment_helpers.get_annotcfg_list(ibs, acfg_name_list)
>>> aid_list = expanded_aids_list[0][0]
>>> ibs.print_annot_stats(aid_list, yawtext_isect=True, per_image=True)
>>> # Expand to get all annots in each chosen image
>>> gid_list = ut.unique_ordered(ibs.get_annot_gids(aid_list))
>>> aid_list = ut.flatten(ibs.get_image_aids(gid_list))
>>> ibs.print_annot_stats(aid_list, yawtext_isect=True, per_image=True)
>>> new_dbpath = ut.get_argval('--new-dbpath', default='PZ_ViewPoints')
>>> new_dbpath = export_annots(ibs, aid_list, new_dbpath)
>>> result = ('new_dbpath = %s' % (str(new_dbpath),))
>>> print(result)

ibeis.dbio.export_subset.export_config_transfer_data(ibs_src, config_rowid_list)[source]¶

ibeis.dbio.export_subset.export_contributor_transfer_data(ibs_src, contributor_rowid, nid_list, species_rowid_list, valid_gid_list=None)[source]¶

CommandLine:

python -m ibeis.dbio.export_subset --test-export_contributor_transfer_data

Example

>>> # SLOW_DOCTEST
>>> import ibeis
>>> from ibeis.dbio import export_subset    # NOQA
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> ibs_src = ibeis.opendb(dbdir='/raid/work2/Turk/PZ_Master')
>>> bulk_conflict_resolution = 'ignore'
>>> gid_list = ibs_src.get_valid_gids()[::10]
>>> user_prompt = False
>>> nid_list = list(set(ut.flatten(ibs_src.get_image_nids(gid_list))))
>>> species_rowid_list = ibs_src._get_all_species_rowids()
>>> contrib_rowid_list = list(set(ibs_src.get_image_contributor_rowid(gid_list)))
>>> valid_gid_list = gid_list
>>> contributor_rowid = contrib_rowid_list[0]
>>> contrib_td = export_contributor_transfer_data(ibs_src, contributor_rowid, nid_list, species_rowid_list, valid_gid_list=valid_gid_list)

Dev::

ibs = ibs_src configid_list = ibs.get_valid_configids() config_suffix_list = ibs.get_config_suffixes(configid_list) print(ut.list_str(list(zip(configid_list, config_suffix_list))))

imgsetid_list = ibs.get_valid_imgsetids() imageset_config_rowid_list = ibs.get_imageset_configid(imgsetid_list) imageset_suffix_list = ibs.get_config_suffixes(config_rowid_list) print(ut.list_str(list(zip(imageset_config_rowid_list, imageset_suffix_list))))

ibeis.dbio.export_subset.export_data(ibs, gid_list, aid_list, nid_list, new_dbpath=None)[source]¶

exports a subset of data and other required info

Parameters:	ibs (IBEISController) – ibeis controller object gid_list (list) – list of image rowids aid_list (list) – list of annotation rowids nid_list (list) – list of name rowids imgsetid_list (list) – list of imageset rowids gsgrid_list (list) – list of imageset-image pairs rowids new_dbpath (None) – (default = None)
Returns:	new_dbpath
Return type:	str

ibeis.dbio.export_subset.export_image_transfer_data(ibs_src, gid_list, config_rowid_list, imgsetid_list, nid_list, species_rowid_list)[source]¶

builds transfer data for seleted image ids in ibs_src. NOTE: gid_list, config_rowid_list and imgsetid_list do not correspond

ibeis.dbio.export_subset.export_images(ibs, gid_list, new_dbpath=None)[source]¶

exports a subset of images and other required info

TODO:

PZ_Master1 needs to backproject information back on to NNP_Master3 and PZ_Master0

Parameters:	ibs (IBEISController) – ibeis controller object gid_list (list) – list of annotation rowids new_dbpath (None) – (default = None)
Returns:	new_dbpath
Return type:	str

ibeis.dbio.export_subset.export_images_temp(ibs)[source]¶

ibeis.dbio.export_subset.export_imageset_transfer_data(ibs_src, imgsetid_list, config_rowid_list)[source]¶

ibeis.dbio.export_subset.export_lblannot_transfer_data(ibs_src, alrid_list, config_rowid_list)[source]¶

ibeis.dbio.export_subset.export_lblimage_transfer_data(ibs_src, glrid_list, config_rowid_list)[source]¶

ibeis.dbio.export_subset.export_name_transfer_data(ibs_src, nid_list)[source]¶

ibeis.dbio.export_subset.export_names(ibs, nid_list, new_dbpath=None)[source]¶

exports a subset of names and other required info

Parameters:	ibs (IBEISController) – ibeis controller object nid_list (list) –

CommandLine:

python -m ibeis.dbio.export_subset --test-export_names

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> import ibeis
>>> # build test data
>>> ibs = ibeis.opendb('testdb2')
>>> ibs.delete_empty_nids()
>>> nid_list = ibs._get_all_known_nids()[0:2]
>>> # execute function
>>> result = export_names(ibs, nid_list)
>>> # verify results
>>> print(result)

ibeis.dbio.export_subset.export_species_transfer_data(ibs_src, species_rowid_list)[source]¶

ibs_src.db.print_schema() print(ibs.db.get_table_csv_header(ibeis.const.SPECIES_TABLE))

ibeis.dbio.export_subset.export_transfer_data(ibs_src, gid_list=None)[source]¶

STEP 1)

Packs all the data you are going to transfer from ibs_src info the transfer_data named tuple.

CommandLine:

python -m ibeis.dbio.export_subset --test-export_transfer_data

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> import ibeis
>>> from ibeis.dbio import export_subset    # NOQA
>>> #ibs_src = ibeis.opendb(dbdir='/raid/work2/Turk/PZ_Master')
>>> ibs_src = ibeis.opendb(db='testdb1')
>>> bulk_conflict_resolution = 'ignore'
>>> num = 5
>>> ibs_src.ensure_contributor_rowids(user_prompt=False)
>>> gid_list = ibs_src.get_valid_gids()[0:num]
>>> td = export_transfer_data(ibs_src, gid_list=gid_list)
>>> assert len(td.contributor_td_list) == 1, 'more than 1 contrib'
>>> contributor_td = td.contributor_td_list[0]
>>> image_td = contributor_td.image_td
>>> assert len(image_td.annotation_td_list) == num
>>> annotation_td = image_td.annotation_td_list[num - 1]
>>> annot_td_dict = annotation_td._asdict()
>>> # remove non-determenistic uuid
>>> del annot_td_dict['annot_uuid_list']
>>> result = ut.dict_str(annot_td_dict)
>>> print(result)
{
    'annot_parent_INDEX_list': [None],
    'annot_theta_list': [0.0],
    'annot_verts_list': [((0, 0), (1072, 0), (1072, 804), (0, 804))],
    'annot_yaw_list': [None],
    'annot_detection_confidence_list': [0.0],
    'annot_exemplar_flag_list': [1],
    'annot_visual_uuid_list': [UUID('5a1a53ba-fd44-b113-7f8c-fcf248d7047f')],
    'annot_semantic_uuid_list': [UUID('02a8b625-fd66-cf7f-8835-468043d0ed63')],
    'annot_note_list': [u''],
    'annot_name_INDEX_list': [1],
    'annot_species_INDEX_list': [0],
    'lblannot_td_list': [None],
}

print(ut.truncate_str(ibs.db.get_table_csv(const.IMAGE_TABLE, exclude_columns=[‘image_uuid’, ‘image_uri’]), 10000))

ibeis.dbio.export_subset.find_gid_list(ibs, min_count=500, ensure_annots=False)[source]¶

ibeis.dbio.export_subset.import_annot_transfer_data(ibs_dst, annot_td, parent_gid, nid_list, species_rowid_list, config_rowid_list)[source]¶

ibeis.dbio.export_subset.import_config_transfer_data(ibs_dst, config_td, contributor_rowid, bulk_conflict_resolution='merge')[source]¶

ibeis.dbio.export_subset.import_contributor_transfer_data(ibs_dst, contributor_td, nid_list, species_rowid_list, bulk_conflict_resolution='merge')[source]¶

ibeis.dbio.export_subset.import_image_transfer_data(ibs_dst, image_td, contributor_rowid, imgsetid_list, nid_list, species_rowid_list, config_rowid_list, bulk_conflict_resolution='merge')[source]¶

ibeis.dbio.export_subset.import_imageset_transfer_data(ibs_dst, imageset_td, config_rowid_list, bulk_conflict_resolution='merge')[source]¶

ibeis.dbio.export_subset.import_lblannot_transfer_data(ibs_dst, lblannot_td, aid, config_rowid_list)[source]¶

ibeis.dbio.export_subset.import_lblimage_transfer_data(ibs_dst, lblimage_td, gid, config_rowid_list)[source]¶

ibeis.dbio.export_subset.import_name_transfer_data(ibs_dst, name_td)[source]¶

ibeis.dbio.export_subset.import_species_transfer_data(ibs_dst, species_td)[source]¶

ibeis.dbio.export_subset.import_transfer_data(ibs_dst, td, bulk_conflict_resolution='merge')[source]¶

Imports transfer data from any ibeis database and moves it into ibs_dst

ibeis.dbio.export_subset.make_new_dbpath(ibs, id_label, id_list)[source]¶

Creates a new database path unique to the exported subset of ids.

ibeis.dbio.export_subset.merge_databases(ibs_src, ibs_dst, gid_list=None, back=None, user_prompt=False, bulk_conflict_resolution='ignore')[source]¶

STEP 0) MAIN DRIVER FUNCTION

Conflict resolutions are only between contributors, configs, imagesets and images. Annotations, lblannots, lblimages, their respective relationships, and image-imageset relationships all inherit the resolution from their associated image.

Parameters:

ibs_src (IBEISController) – source controller ibs_dst (IBEISController) – destination controller (can be an empty database) back (GUIBackend) – optional gui to update user_prompt (bool) – prompt user for information bulk_conflict_resolution (str) – valid conflict_resolutions are:: +— * ‘replace’ - delete original in ibs_dst and import new value * ‘ignore’ - ignore imported value and keep original in ibs_dst * ‘merge’ - (default) keep both values in both databases +— WARNING - this may cause near-duplicate information due to duplicate detections, duplicate manual annotations from different contributors. Images and lblimages will not be duplicated. WARNING - this will only keep the meta data for an image, annotation from the source database (image_height, annotation_verts, etc.) WARNING - this may cause an exception to be raised +—

CommandLine:

python -m ibeis.dbio.export_subset --test-merge_databases
python -m ibeis.dbio.export_subset --test-merge_databases:1

Example

>>> # SLOW_DOCTEST
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> import ibeis
>>> from ibeis.dbio import export_subset
>>> ibs_src = ibeis.opendb(db='testdb1')
>>> bulk_conflict_resolution = 'ignore'
>>> #gid_list = None
>>> back = None
>>> user_prompt = False
>>> #ibs_src2 = ibeis.opendb(dbdir='PZ_MTEST')
>>> print(ibs_src.get_infostr())
>>> #print(ibs_src2.get_infostr())
>>> # OPEN A CLEAN DATABASE
>>> ibs_dst = ibeis.opendb(dbdir='testdb_dst', allow_newdir=True, delete_ibsdir=True)
>>> assert ibs_dst.get_num_names() == 0, 'dst database is not empty'
>>> assert ibs_dst.get_num_images() == 0, 'dst database is not empty'
>>> assert ibs_dst.get_num_annotations() == 0, 'dst database is not empty'
>>> #ibs_dst = ibs
>>> gid_list = ibs_src.get_valid_gids()
>>> print('Execute test func')
>>> export_subset.merge_databases(ibs_src, ibs_dst, gid_list,
...                               bulk_conflict_resolution=bulk_conflict_resolution)
>>> #export_subset.merge_databases(ibs_src2, ibs_dst, bulk_conflict_resolution='ignore')
>>> result = ibs_dst.get_infostr()
>>> print('Result:')
>>> print(result)
dbname = 'testdb_dst'
num_images = 13
num_annotations = 13
num_names = 7

Example2:

>>> # SLOW_DOCTEST
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> import ibeis
>>> from ibeis.dbio import export_subset
>>> ibs_src = ibeis.opendb(db='PZ_MTEST')
>>> bulk_conflict_resolution = 'ignore'
>>> #gid_list = None
>>> back = None
>>> user_prompt = False
>>> #ibs_src2 = ibeis.opendb(dbdir='PZ_MTEST')
>>> print(ibs_src.get_infostr())
>>> #print(ibs_src2.get_infostr())
>>> # OPEN A CLEAN DATABASE
>>> ibs_dst = ibeis.opendb(dbdir='testdb_dst', allow_newdir=False, delete_ibsdir=False)
>>> assert ibs_dst.get_num_names() > 0, 'dst database is empty'
>>> assert ibs_dst.get_num_images() > 0, 'dst database is empty'
>>> assert ibs_dst.get_num_annotations() > 0, 'dst database is empty'
>>> #ibs_dst = ibs
>>> gid_list = ibs_src.get_valid_gids()
>>> print('Execute test func')
>>> export_subset.merge_databases(ibs_src, ibs_dst, gid_list,
...                               bulk_conflict_resolution=bulk_conflict_resolution)
>>> #export_subset.merge_databases(ibs_src2, ibs_dst, bulk_conflict_resolution='ignore')
>>> result = ibs_dst.get_infostr()
>>> print('Result:')
>>> print(result)
dbname = 'testdb_dst'
num_images = 128
num_annotations = 129
num_names = 48

ibeis.dbio.export_subset.merge_databases2(ibs_src, ibs_dst, rowid_subsets=None)[source]¶

New way of merging using the non-hacky sql table merge. However, its only workings due to major hacks.

FIXME: annotmatch table

CommandLine:

python -m ibeis merge_databases2

python -m ibeis.dbio.export_subset --test-merge_databases2:0
python -m ibeis.dbio.export_subset --test-merge_databases2:0 --db1 PZ_Master0 --db2 PZ_Master1
python -m ibeis.dbio.export_subset --test-merge_databases2:0 --db1 NNP_Master3 --db2 PZ_Master1

python -m ibeis.dbio.export_subset --test-merge_databases2:0 --db1 GZ_ALL --db2 GZ_Master1
python -m ibeis.dbio.export_subset --test-merge_databases2:0 --db1 lewa_grevys --db2 GZ_Master1

Example

>>> # ENABLE_DOCTEST
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> import ibeis
>>> db1 = ut.get_argval('--db1', str, default=None)
>>> db2 = ut.get_argval('--db2', str, default=None)
>>> dbdir1 = ut.get_argval('--dbdir1', str, default=None)
>>> dbdir2 = ut.get_argval('--dbdir2', str, default=None)
>>> delete_ibsdir = False
>>> # Check for test mode instead of script mode
>>> if db1 is None and db2 is None and dbdir1 is None and dbdir2 is None:
...     db1 = 'testdb1'
...     dbdir2 = 'testdb_dst'
...     delete_ibsdir = True
>>> # Open the source and destination database
>>> assert db1 is not None or dbdir1 is not None
>>> assert db2 is not None or dbdir2 is not None
>>> ibs_src = ibeis.opendb(db=db1, dbdir=dbdir1)
>>> ibs_dst = ibeis.opendb(db=db2, dbdir=dbdir2, allow_newdir=True, delete_ibsdir=delete_ibsdir)
>>> merge_databases2(ibs_src, ibs_dst)
>>> check_merge(ibs_src, ibs_dst)
>>> ibs_dst.print_dbinfo()

ibeis.dbio.export_subset.test_merge()[source]¶

CommandLine:

python -m ibeis.dbio.export_subset --test-test_merge

Example

>>> # SLOW_DOCTEST
>>> from ibeis.dbio.export_subset import *  # NOQA
>>> result = test_merge()
>>> print(result)

ibeis.dbio.export_subset.tryindex(value, list_, warning=True)[source]¶

ibeis.dbio.ingest_database module¶

This module lists known raw databases and how to ingest them.

Specify arguments and run the following command to ingest a database

python -m ibeis –tf ingest_rawdata –db seaturtles –imgdir “~/turtles/Turtles from Jill” –ingest-type=named_folders –species=turtles

# — GET DATA — rsync -avhzP <user>@<host>:<remotedir> <path-to-raw-imgs> # — RUN INGEST SCRIPT — python -m ibeis –tf ingest_rawdata –db <new-ibeis-db-name> –imgdir <path-to-raw-imgs> –ingest-type=named_folders –species=<optional> –fmtkey=<optional>

class ibeis.dbio.ingest_database.FMT_KEYS[source]¶

Bases: object

elephant_fmt = '{prefix?}{name}_{view}_{id?}.{ext}'¶

giraffe1_fmt = '{name:*}_{id:d}.{ext}'¶

name_fmt = '{name:*}[id:d].{ext}'¶

seal2_fmt = '{name:Phsd*}{id:[A-Z]}.{ext}'¶

snails_fmt = '{name:*dd}{id:dd}.{ext}'¶

class ibeis.dbio.ingest_database.Ingestable(dbname, img_dir=None, ingest_type=None, fmtkey=None, adjust_percent=0.0, postingest_func=None, zipfile=None, species=None, images_as_annots=True)[source]¶

Bases: object

Temporary structure representing how to ingest a databases

ensure_feasibility()[source]¶

class ibeis.dbio.ingest_database.Ingestable2(dbdir, imgpath_list=None, imgdir_list=None, zipfile_list=None, postingest_func=None, ingest_config={}, **kwargs)[source]¶

Bases: object

execute(ibs=None)[source]¶

ibeis.dbio.ingest_database.get_name_texts_from_gnames(gpath_list, img_dir, fmtkey='{name:*}[aid:d].{ext}')[source]¶

Parameters:	gpath_list (list) – list of image paths img_dir (str) – path to image directory fmtkey (str) – pattern string to parse names from (default = ‘{name:*}[aid:d].{ext}’)
Returns:	name_list - based on the parent folder of each image
Return type:	list

CommandLine:

python -m ibeis.dbio.ingest_database --test-get_name_texts_from_gnames

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> gpath_list = ['e_f0273_f.jpg', 'f0001_f.jpg', 'f0259_l_3.jpg', 'f0259_f_1.jpg',  'f0259_f (1).jpg', 'f0058_u16_f.jpg']
>>> img_dir = ''
>>> fmtkey = FMT_KEYS.elephant_fmt
>>> result = get_name_texts_from_gnames(gpath_list, img_dir, fmtkey)
>>> print(result)

ibeis.dbio.ingest_database.get_name_texts_from_parent_folder(gpath_list, img_dir, fmtkey=None)[source]¶

Input: gpath_list Output: names based on the parent folder of each image

ibeis.dbio.ingest_database.get_standard_ingestable(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_Elephants_drop1(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_Giraffes1(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_JAG_Kieryn(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_humpbacks(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_lynx(dbname)[source]¶

CommandLine:

python -m ibeis.dbio.ingest_database --exec-injest_main --db lynx

ibeis.dbio.ingest_database.ingest_oxford_style_db(dbdir, dryrun=False)[source]¶

Ingest either oxford or paris

Parameters:	dbdir (str) –

CommandLine:

python -m ibeis.dbio.ingest_database --exec-ingest_oxford_style_db --show

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> dbdir = '/raid/work/Oxford'
>>> dryrun = True
>>> ingest_oxford_style_db(dbdir)
>>> ut.quit_if_noshow()
>>> import plottool as pt
>>> ut.show_if_requested()

ibeis.dbio.ingest_database.ingest_polar_bears(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_rawdata(ibs, ingestable, localize=False)[source]¶

Ingests rawdata into an ibeis database.

Parameters:	ibs (ibeis.IBEISController) – ibeis controller object ingestable (Ingestable) – localize (bool) – (default = False)
Returns:	aid_list - list of annotation rowids
Return type:	list

Notes

if ingest_type == ‘named_folders’:

Converts folder structure where folders = name, to ibsdb

if ingest_type == ‘named_images’:

Converts imgname structure where imgnames = name_id.ext, to ibsdb

CommandLine:

python ibeis/dbio/ingest_database.py --db seals_drop2
python -m ibeis.dbio.ingest_database --exec-ingest_rawdata
python -m ibeis.dbio.ingest_database --exec-ingest_rawdata --db snow-leopards --imgdir /raid/raw_rsync/snow-leopards

python -m ibeis --tf ingest_rawdata --db wd_peter2 --imgdir /raid/raw_rsync/african-dogs --ingest-type=named_folders --species=wild_dog --fmtkey='African Wild Dog: {name}' --force-delete
python -m ibeis --tf ingest_rawdata --db <newdbname>  --imgdir <path-to-images> --ingest-type=named_folders --species=humpback

Example

>>> # SCRIPT
>>> # General ingest script
>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> import ibeis
>>> dbname = ut.get_argval('--db', str, None)  # 'snow-leopards')
>>> force_delete = ut.get_argflag(('--force_delete', '--force-delete'))
>>> img_dir = ut.get_argval('--imgdir', type_=str, default=None)
>>> ingest_type = ut.get_argval('--ingest-type', type_=str, default='unknown')
>>> fmtkey = ut.get_argval('--fmtkey', type_=str, default=None)
>>> species = ut.get_argval('--species', type_=str, default=None)
>>> assert img_dir is not None, 'specify img dir'
>>> assert dbname is not None, 'specify dbname'
>>> ingestable = Ingestable(
>>>     dbname, img_dir=img_dir, ingest_type=ingest_type,
>>>     fmtkey=fmtkey, species=species, images_as_annots=ingest_type != 'unknown',
>>>     adjust_percent=0.00)
>>> from ibeis.control import IBEISControl
>>> dbdir = ibeis.sysres.db_to_dbdir(dbname, allow_newdir=True, use_sync=False)
>>> ut.ensuredir(dbdir, verbose=True)
>>> if force_delete:
>>>     ibsfuncs.delete_ibeis_database(dbdir)
>>> ibs = IBEISControl.request_IBEISController(dbdir)
>>> localize = False
>>> aid_list = ingest_rawdata(ibs, ingestable, localize)
>>> result = ('aid_list = %s' % (str(aid_list),))
>>> print(result)

ibeis.dbio.ingest_database.ingest_seals_drop2(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_serengeti_mamal_cameratrap(species)[source]¶

Downloads data from Serengeti dryad server

References

http://datadryad.org/resource/doi:10.5061/dryad.5pt92 Swanson AB, Kosmala M, Lintott CJ, Simpson RJ, Smith A, Packer C (2015) Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna. Scientific Data 2: 150026. http://dx.doi.org/10.1038/sdata.2015.26 Swanson AB, Kosmala M, Lintott CJ, Simpson RJ, Smith A, Packer C (2015) Data from: Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna. Dryad Digital Repository. http://dx.doi.org/10.5061/dryad.5pt92

Parameters:	species –

CommandLine:

python -m ibeis.dbio.ingest_database --test-ingest_serengeti_mamal_cameratrap --species zebra_plains
python -m ibeis.dbio.ingest_database --test-ingest_serengeti_mamal_cameratrap --species cheetah

Example

>>> # SCRIPT
>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> import ibeis
>>> species = ut.get_argval('--species', type_=str, default=ibeis.const.TEST_SPECIES.ZEB_PLAIN)
>>> # species = ut.get_argval('--species', type_=str, default='cheetah')
>>> result = ingest_serengeti_mamal_cameratrap(species)
>>> print(result)

ibeis.dbio.ingest_database.ingest_snails_drop1(dbname)[source]¶

ibeis.dbio.ingest_database.ingest_standard_database(dbname, force_delete=False)[source]¶

Parameters:	dbname (str) – database name force_delete (bool) –

Example

>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> dbname = 'testdb1'
>>> force_delete = False
>>> result = ingest_standard_database(dbname, force_delete)
>>> print(result)

ibeis.dbio.ingest_database.ingest_testdb1(dbname)[source]¶

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> import utool as ut
>>> from vtool.tests import grabdata
>>> import ibeis
>>> grabdata.ensure_testdata()
>>> # DELETE TESTDB1
>>> TESTDB1 = ut.unixjoin(ibeis.sysres.get_workdir(), 'testdb1')
>>> ut.delete(TESTDB1, ignore_errors=False)
>>> result = ingest_testdb1(dbname)

ibeis.dbio.ingest_database.ingest_whale_sharks(dbname)[source]¶

CommandLine:

python -m ibeis.dbio.ingest_database --exec-injest_main --db WS_ALL

ibeis.dbio.ingest_database.ingest_wilddog_peter(dbname)[source]¶

CommandLine:

python -m ibeis.dbio.ingest_database --exec-injest_main --db wd_peter_blinston

ibeis.dbio.ingest_database.injest_main()[source]¶

CommandLine:

python -m ibeis.dbio.ingest_database --test-injest_main
python -m ibeis.dbio.ingest_database --test-injest_main --db snow-leopards

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.dbio.ingest_database import *  # NOQA
>>> injest_main()

ibeis.dbio.ingest_database.normalize_name(name)[source]¶

Maps unknonwn names to the standard ____

ibeis.dbio.ingest_database.resolve_name_conflicts(gid_list, name_list)[source]¶

ibeis.dbio.ingest_hsdb module¶

Converts a hotspostter database to IBEIS

ibeis.dbio.ingest_hsdb.check_unconverted_hsdb(dbdir)[source]¶

Returns if a directory is an unconverted hotspotter database

ibeis.dbio.ingest_hsdb.convert_hsdb_to_ibeis(hsdb_dir, **kwargs)[source]¶

Parameters:	hsdb_dir (str) –

CommandLine:

python -m ibeis.dbio.ingest_hsdb --test-convert_hsdb_to_ibeis:0 --db ~/work/Frogs

Example

>>> # SCRIPT
>>> from ibeis.dbio.ingest_hsdb import *  # NOQA
>>> hsdb_dir = ut.get_argval('--dbdir', type_=str, default=None)
>>> result = convert_hsdb_to_ibeis(hsdb_dir)
>>> print(result)

ibeis.dbio.ingest_hsdb.get_hsinternal(hsdb_dir)[source]¶

ibeis.dbio.ingest_hsdb.get_unconverted_hsdbs(workdir=None)[source]¶

Parameters:	workdir (None) – (default = None)

CommandLine:

python -m ibeis.dbio.ingest_hsdb --test-get_unconverted_hsdbs

Example

>>> # SCRIPT
>>> from ibeis.dbio.ingest_hsdb import *  # NOQA
>>> workdir = None
>>> result = get_unconverted_hsdbs(workdir)
>>> print(result)

ibeis.dbio.ingest_hsdb.ingest_unconverted_hsdbs_in_workdir()[source]¶

ibeis.dbio.ingest_hsdb.is_hsdb(dbdir)[source]¶

ibeis.dbio.ingest_hsdb.is_hsdbv3(dbdir)[source]¶

ibeis.dbio.ingest_hsdb.is_hsdbv4(dbdir)[source]¶

ibeis.dbio.ingest_hsdb.is_hsinternal(dbdir)[source]¶

ibeis.dbio.ingest_hsdb.is_succesful_convert(dbdir)[source]¶

the sucess flag is only written if the _ibsdb was properly generated

ibeis.dbio.ingest_hsdb.test_open_to_convert()[source]¶

CommandLine:

python -m ibeis.dbio.ingest_hsdb --test-test_open_to_convert

Example

>>> # VERY_UNSTABLE_DOCTEST
>>> from ibeis.dbio.ingest_hsdb import *  # NOQA
>>> result = test_open_to_convert()
>>> print(result)

ibeis.dbio.ingest_hsdb.testdata_ensure_unconverted_hsdb()[source]¶

Makes an unconverted test datapath

CommandLine:

python -m ibeis.dbio.ingest_hsdb --test-testdata_ensure_unconverted_hsdb

Example

>>> # SCRIPT
>>> from ibeis.dbio.ingest_hsdb import *  # NOQA
>>> result = testdata_ensure_unconverted_hsdb()
>>> print(result)

ibeis.dbio.ingest_mdb module¶

Module contents¶

this module handles importing and exporting. the best word i can think of is io. maybe marshall?