ibeis.web package¶

Submodules¶

ibeis.web.apis module¶

Dependencies: flask, tornado

ibeis.web.apis.heartbeat(*args, **kwargs)[source]¶

ibeis.web.apis.hello_world(*args, **kwargs)[source]¶

CommandLine:

python -m ibeis.web.apis --exec-hello_world:0
python -m ibeis.web.apis --exec-hello_world:1

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.app import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web(browser=True, url_suffix='/api/test/helloworld/?test0=0')  # start_job_queue=False)
>>> print('Server will run until control c')
>>> #web_ibs.terminate2()

Example1:

>>> # WEB_DOCTEST
>>> from ibeis.web.app import *  # NOQA
>>> import ibeis
>>> import requests
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web('testdb1', start_job_queue=False)
>>> domain = 'http://127.0.0.1:5000'
>>> url = domain + '/api/test/helloworld/?test0=0'
>>> payload = {
>>>     'test1' : 'test1',
>>>     'test2' : None,  # NOTICE test2 DOES NOT SHOW UP
>>> }
>>> resp = requests.post(url, data=payload)
>>> print(resp)
>>> web_ibs.terminate2()

ibeis.web.apis.image_src_api(gid=None, thumbnail=False, fresh=False, **kwargs)[source]¶

Returns the image file of image <gid>

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.app import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web('testdb1', start_job_queue=False)
>>> web_ibs.send_ibeis_request('/api/image/src/', type_='get', gid=1)
>>> print(resp)
>>> web_ibs.terminate2()

RESTful:: Method: GET URL: /api/image/src/<gid>/

ibeis.web.apis.image_src_api_json(image_uuid=None, **kwargs)[source]¶

Returns the image file of image <gid>

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.app import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web('testdb1', start_job_queue=False)
>>> web_ibs.send_ibeis_request('/api/image/src/', type_='get', gid=1)
>>> print(resp)
>>> web_ibs.terminate2()

RESTful:: Method: GET URL: /api/image/src/<gid>/

ibeis.web.apis.image_upload(cleanup=True, **kwargs)[source]¶

Returns the gid for an uploaded image.

Parameters:	image (image binary) – the POST variable containing the binary (multi-form) image data **kwargs – Arbitrary keyword arguments; the kwargs are passed down to the add_images function
Returns:	gid – gid corresponding to the image submitted. lexigraphical order.
Return type:	rowids

RESTful:: Method: POST URL: /api/image/

ibeis.web.apis.image_upload_zip(**kwargs)[source]¶

Returns the gid_list for image files submitted in a ZIP archive. The image archive should be flat (no folders will be scanned for images) and must be smaller than 100 MB. The archive can submit multiple images, ideally in JPEG format to save space. Duplicate image uploads will result in the duplicate images receiving the same gid based on the hashed pixel values.

Parameters:	image_zip_archive (binary) – the POST variable containing the binary (multi-form) image archive data **kwargs – Arbitrary keyword arguments; the kwargs are passed down to the add_images function
Returns:	gid_list – the list of gids corresponding to the images submitted. The gids correspond to the image names sorted in lexigraphical order.
Return type:	list if rowids

RESTful:: Method: POST URL: /api/image/zip

ibeis.web.apis_detect module¶

Dependencies: flask, tornado

ibeis.web.apis_detect.commit_detection_results(ibs, gid_list, results_list, note=None)[source]¶

ibeis.web.apis_detect.commit_localization_results(ibs, gid_list, results_list, note=None)[source]¶

ibeis.web.apis_detect.detect_cnn_yolo(ibs, gid_list, commit=True, **kwargs)[source]¶

Runs animal detection in each image. Adds annotations to the database as they are found.

Parameters:	gid_list (list) – list of image ids to run detection on
Returns:	aids_list – list of lists of annotation ids detected in each image
Return type:	list

CommandLine:

python -m ibeis.web.apis_detect --test-detect_cnn_yolo --show

RESTful:: Method: PUT, GET URL: /api/detect/cnn/yolo/

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_detect import *  # NOQA
>>> import ibeis
>>> # build test data
>>> ibs = ibeis.opendb('PZ_MTEST')
>>> gid_list = ibs.get_valid_gids()[:5]
>>> # execute function
>>> aids_list = ibs.detect_cnn_yolo(gid_list)
>>> # Visualize results
>>> if ut.show_was_requested():
>>>     import plottool as pt
>>>     from ibeis.viz import viz_image
>>>     for fnum, gid in enumerate(gid_list):
>>>         viz_image.show_image(ibs, gid, fnum=fnum)
>>>     pt.show_if_requested()
>>> # Remove newly detected annotations
>>> ibs.delete_annots(ut.flatten(aids_list))

ibeis.web.apis_detect.detect_cnn_yolo_json(ibs, gid_list, **kwargs)[source]¶

Runs animal detection in each image and returns json-ready formatted: results, does not return annotations

Parameters:	gid_list (list) – list of image ids to run detection on
Returns:	results_dict – dict of detection results (not annotations)
Return type:	list

CommandLine:

python -m ibeis.web.apis_detect --test-detect_cnn_yolo_json

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_detect import *  # NOQA
>>> import ibeis
>>> # build test data
>>> ibs = ibeis.opendb('testdb1')
>>> gid_list = ibs.get_valid_gids()[0:2]
>>> # execute function
>>> results_dict = ibs.detect_cnn_yolo_json(gid_list)
>>> print(results_dict)

ibeis.web.apis_detect.detect_random_forest(ibs, gid_list, species, commit=True, **kwargs)[source]¶

Runs animal detection in each image. Adds annotations to the database as they are found.

Parameters:	gid_list (list) – list of image ids to run detection on species (str) – string text of the species to identify
Returns:	aids_list – list of lists of annotation ids detected in each image
Return type:	list

CommandLine:

python -m ibeis.web.apis_detect --test-detect_random_forest --show

RESTful:: Method: PUT, GET URL: /api/detect/random_forest/

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_detect import *  # NOQA
>>> import ibeis
>>> # build test data
>>> ibs = ibeis.opendb('testdb1')
>>> gid_list = ibs.get_valid_gids()[0:2]
>>> species = ibeis.const.TEST_SPECIES.ZEB_PLAIN
>>> # execute function
>>> aids_list = ibs.detect_random_forest(gid_list, species)
>>> # Visualize results
>>> if ut.show_was_requested():
>>>     import plottool as pt
>>>     from ibeis.viz import viz_image
>>>     for fnum, gid in enumerate(gid_list):
>>>         viz_image.show_image(ibs, gid, fnum=fnum)
>>>     pt.show_if_requested()
>>> # Remove newly detected annotations
>>> ibs.delete_annots(ut.flatten(aids_list))

ibeis.web.apis_detect.detection_yolo_test(ibs)[source]¶

ibeis.web.apis_detect.get_species_with_detectors(ibs)[source]¶

RESTful:: Method: GET URL: /api/detect/species/

ibeis.web.apis_detect.get_working_species(ibs)[source]¶

RESTful:: Method: GET URL: /api/detect/species/working/

ibeis.web.apis_detect.has_species_detector(ibs, species_text)[source]¶

TODO: extend to use non-constant species

RESTful:: Method: GET URL: /api/detect/species/enabled/

ibeis.web.apis_detect.process_detection_html(ibs, **kwargs)[source]¶

Processes the return from the detection review interface. Pass the POST result from the detection review form directly to this function unmodified

Returns:	detection results – Same format as func:start_detect_image
Return type:	dict

RESTful:: Method: POST URL: /api/review/detect/cnn/yolo/

ibeis.web.apis_detect.review_detection_html(ibs, image_uuid, result_list, callback_url, callback_method='POST', include_jquery=False)[source]¶

Returns the detection review interface for a particular image UUID and a list of results for that image.

Parameters:	image_uuid (UUID) – the UUID of the image you want to review detections for result_list (list of dict) – list of detection results returned by the detector callback_url (str) – URL that the review form will submit to (action) when the user is complete with their review callback_method (str) – HTTP method the review form will submit to (method). Defaults to ‘POST’
Returns:	template – json response with the detection web interface in html
Return type:	html

RESTful:: Method: GET URL: /api/review/detect/cnn/yolo/

ibeis.web.apis_detect.review_detection_test()[source]¶

ibeis.web.apis_engine module¶

ibeis.web.apis_engine.ensure_simple_server(port=5832)[source]¶

CommandLine:

python -m ibeis.web.apis_engine --exec-ensure_simple_server
python -m utool.util_web --exec-start_simple_webserver

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_engine import *  # NOQA
>>> result = ensure_simple_server()
>>> print(result)

ibeis.web.apis_engine.start_detect_image(ibs, image_uuid_list, callback_url=None, callback_method=None)[source]¶

REST:: Method: GET URL: /api/engine/detect/cnn/yolo/

Parameters:	image_uuid_list (list) – list of image uuids to detect on. callback_url (url) – url that will be called when detection succeeds or fails

ibeis.web.apis_engine.start_identify_annots(ibs, qannot_uuid_list, dannot_uuid_list=None, pipecfg={}, callback_url=None, callback_method=None)[source]¶

REST:: Method: GET URL: /api/engine/start_identify_annots/

Parameters:	qannot_uuid_list (list) – specifies the query annotations to identify. dannot_uuid_list (list) – specifies the annotations that the algorithm is allowed to use for identification. If not specified all annotations are used. (default=None) pipecfg (dict) – dictionary of pipeline configuration arguments (default=None)

CommandLine:

# Run as main process
python -m ibeis.web.apis_engine --exec-start_identify_annots:0
# Run using server process
python -m ibeis.web.apis_engine --exec-start_identify_annots:1

# Split into multiple processes
python -m ibeis.web.apis_engine --main --bg
python -m ibeis.web.apis_engine --exec-start_identify_annots:1 --fg

python -m ibeis.web.apis_engine --exec-start_identify_annots:1 --domain http://52.33.105.88

python -m ibeis.web.apis_engine --exec-start_identify_annots:1 --duuids=[]
python -m ibeis.web.apis_engine --exec-start_identify_annots:1 --domain http://52.33.105.88 --duuids=03a17411-c226-c960-d180-9fafef88c880

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_engine import *  # NOQA
>>> from ibeis.web import apis_engine
>>> import ibeis
>>> ibs, qaids, daids = ibeis.testdata_expanded_aids(
>>>     defaultdb='PZ_MTEST', a=['default:qsize=2,dsize=10'])
>>> qannot_uuid_list = ibs.get_annot_uuids(qaids)
>>> dannot_uuid_list = ibs.get_annot_uuids(daids)
>>> pipecfg = {}
>>> ibs.initialize_job_manager()
>>> jobid = ibs.start_identify_annots(qannot_uuid_list, dannot_uuid_list, pipecfg)
>>> result = ibs.wait_for_job_result(jobid, timeout=None, freq=2)
>>> print(result)
>>> import utool as ut
>>> #print(ut.to_json(result))
>>> ibs.close_job_manager()

Example

>>> from ibeis.web.apis_engine import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb('testdb1')  # , domain='http://52.33.105.88')
>>> aids = ibs.get_valid_aids()[0:2]
>>> qaids = aids[0:1]
>>> daids = aids
>>> query_config_dict = {
>>>     #'pipeline_root' : 'BC_DTW'
>>> }
>>> cm_list = ibs.query_chips(qaids, daids, cfgdict=query_config_dict)

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.apis_engine import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web('testdb1')  # , domain='http://52.33.105.88')
>>> aids = web_ibs.send_ibeis_request('/api/annot/', 'get')[0:2]
>>> uuid_list = web_ibs.send_ibeis_request('/api/annot/uuids/', type_='get', aid_list=aids)
>>> quuid_list = ut.get_argval('--quuids', type_=list, default=uuid_list)
>>> duuid_list = ut.get_argval('--duuids', type_=list, default=uuid_list)
>>> data = dict(
>>>     qannot_uuid_list=quuid_list, dannot_uuid_list=duuid_list,
>>>     pipecfg={},
>>>     callback_url='http://127.0.1.1:5832'
>>> )
>>> # Start callback server
>>> bgserver = ensure_simple_server()
>>> # --
>>> jobid = web_ibs.send_ibeis_request('/api/engine/start_identify_annots/', **data)
>>> status_response = web_ibs.wait_for_results(jobid, delays=[1, 5, 30])
>>> print('status_response = %s' % (status_response,))
>>> result_response = web_ibs.read_engine_results(jobid)
>>> print('result_response = %s' % (result_response,))
>>> cm_dict = result_response['json_result'][0]
>>> print('Finished test')
>>> web_ibs.terminate2()
>>> bgserver.terminate2()

ibeis.web.apis_engine.start_identify_annots_query(ibs, query_annot_uuid_list=None, query_annot_name_list=None, database_annot_uuid_list=None, database_annot_name_list=None, matching_state_list=[], query_config_dict={}, echo_query_params=True, callback_url=None, callback_method=None)[source]¶

REST:: Method: GET URL: /api/engine/query/graph/

Parameters:

query_annot_uuid_list (list) – specifies the query annotations to identify.
query_annot_name_list (list) – specifies the query annotation names
database_annot_uuid_list (list) – specifies the annotations that the algorithm is allowed to use for identification. If not specified all annotations are used. (default=None)
database_annot_name_list (list) – specifies the database annotation names (default=None)
matching_state_list (list of tuple) – the list of matching state 3-tuples corresponding to the query_annot_uuid_list (default=None)
query_config_dict (dict) – dictionary of algorithmic configuration arguments. (default=None)
echo_query_params (bool) – flag for if to return the original query parameters with the result

CommandLine:

# Normal mode
python -m ibeis.web.apis_engine start_identify_annots_query
# Split mode
ibeis --web
python -m ibeis.web.apis_engine start_identify_annots_query --show --domain=localhost

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_engine import *  # NOQA
>>> import ibeis
>>> #domain = 'localhost'
>>> domain = None
>>> web_ibs = ibeis.opendb_bg_web('testdb1', domain=domain)  # , domain='http://52.33.105.88')
>>> aids = web_ibs.send_ibeis_request('/api/annot/', 'get')[0:3]
>>> uuid_list = web_ibs.send_ibeis_request('/api/annot/uuids/', type_='get', aid_list=aids)
>>> quuid_list = ut.get_argval('--quuids', type_=list, default=uuid_list)[0:1]
>>> duuid_list = ut.get_argval('--duuids', type_=list, default=uuid_list)
>>> query_config_dict = {
>>>    #'pipeline_root' : 'BC_DTW'
>>> }
>>> data = dict(
>>>     query_annot_uuid_list=quuid_list, database_annot_uuid_list=duuid_list,
>>>     query_config_dict=query_config_dict,
>>> )
>>> jobid = web_ibs.send_ibeis_request('/api/engine/query/graph/', **data)
>>> print('jobid = %r' % (jobid,))
>>> status_response = web_ibs.wait_for_results(jobid)
>>> result_response = web_ibs.read_engine_results(jobid)
>>> print('result_response = %s' % (ut.repr3(result_response),))
>>> inference_result = result_response['json_result']
>>> if isinstance(inference_result, six.string_types):
>>>    print(inference_result)
>>> cm_dict = inference_result['cm_dict']
>>> quuid = quuid_list[0]
>>> cm = cm_dict[str(quuid)]
>>> web_ibs.terminate2()

ibeis.web.apis_engine.web_check_uuids(ibs, image_uuid_list=[], qannot_uuid_list=[], dannot_uuid_list=[])[source]¶

Parameters:	ibs (ibeis.IBEISController) – image analysis api image_uuid_list (list) – (default = []) qannot_uuid_list (list) – (default = []) dannot_uuid_list (list) – (default = [])

CommandLine:

python -m ibeis.web.apis_engine --exec-web_check_uuids --show

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.apis_engine import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> image_uuid_list = []
>>> qannot_uuid_list = ibs.get_annot_uuids([1, 1, 2, 3, 2, 4])
>>> dannot_uuid_list = ibs.get_annot_uuids([1, 2, 3])
>>> try:
>>>     web_check_uuids(ibs, image_uuid_list, qannot_uuid_list,
>>>                     dannot_uuid_list)
>>> except controller_inject.DuplicateUUIDException:
>>>     pass
>>> else:
>>>     raise AssertionError('Should have gotten DuplicateUUIDException')
>>> try:
>>>     web_check_uuids(ibs, [1, 2, 3], qannot_uuid_list,
>>>                     dannot_uuid_list)
>>> except controller_inject.WebMissingUUIDException as ex:
>>>     pass
>>> else:
>>>     raise AssertionError('Should have gotten WebMissingUUIDException')
>>> print('Successfully reported errors')

ibeis.web.apis_json module¶

Dependencies: flask, tornado

ibeis.web.apis_json.add_annots_json(ibs, image_uuid_list, annot_uuid_list, annot_bbox_list, annot_theta_list=None, annot_species_list=None, annot_name_list=None, annot_notes_list=None, **kwargs)[source]¶

REST:: Method: POST URL: /api/annot/json/

Parameters:

image_uuid_list (list of str) – list of image UUIDs to be used in IBEIS IA
annot_uuid_list (list of str) – list of annotations UUIDs to be used in IBEIS IA
annot_bbox_list (list of 4-tuple) – list of bounding box coordinates encoded as a 4-tuple of the values (xtl, ytl, width, height) where xtl is the ‘top left corner, x value’ and ytl is the ‘top left corner, y value’.
annot_theta_list (list of float) – list of radian rotation around center. Defaults to 0.0 (no rotation).
annot_species_list (list of str) – list of species for the annotation, if known. If the list is partially known, use None (null in JSON) for unknown entries.
annot_name_list (list of str) – list of names for the annotation, if known. If the list is partially known, use None (null in JSON) for unknown entries.
annot_notes_list (list of str) – list of notes to be added to the annotation.
**kwargs – key-value pairs passed to the ibs.add_annots() function.

CommandLine:

python -m ibeis.web.app --test-add_annots_json

Example

>>> import ibeis
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> web_instance = ibeis.opendb(db='testdb1')
>>> _payload = {
>>>     'image_uuid_list': [
>>>         uuid.UUID('7fea8101-7dec-44e3-bf5d-b8287fd231e2'),
>>>         uuid.UUID('c081119a-e08e-4863-a710-3210171d27d6'),
>>>     ],
>>>     'annot_uuid_list': [
>>>         uuid.UUID('fe1547c5-1425-4757-9b8f-b2b4a47f552d'),
>>>         uuid.UUID('86d3959f-7167-4822-b99f-42d453a50745'),
>>>     ],
>>>     'annot_bbox_list': [
>>>         [0, 0, 1992, 1328],
>>>         [0, 0, 1194, 401],
>>>     ],
>>> }
>>> aid_list = ibeis.web.app.add_annots_json(web_instance, **_payload)
>>> print(aid_list)
>>> print(web_instance.get_annot_image_uuids(aid_list))
>>> print(web_instance.get_annot_uuids(aid_list))
>>> print(web_instance.get_annot_bboxes(aid_list))

ibeis.web.apis_json.add_images_json(ibs, image_uri_list, image_uuid_list, image_width_list, image_height_list, image_orig_name_list=None, image_ext_list=None, image_time_posix_list=None, image_gps_lat_list=None, image_gps_lon_list=None, image_orientation_list=None, image_notes_list=None, **kwargs)[source]¶

REST:: Method: POST URL: /api/image/json/

Parameters:

image_uri_list (list) –
list of string image uris, most likely HTTP(S) or S3 encoded URLs. Alternatively, this can be a list of dictionaries (JSON objects) that specify AWS S3 stored assets. An example below:

image_uri_list = [

‘http://domain.com/example/asset1.png‘, ‘/home/example/Desktop/example/asset2.jpg’, ‘s3://s3.amazon.com/example-bucket-2/asset1-in-bucket-2.tif’, {

‘bucket’ : ‘example-bucket-1’, ‘key’ : ‘example/asset1.png’, ‘auth_domain’ : None, # Uses localhost ‘auth_access_id’ : None, # Uses system default ‘auth_secret_key’ : None, # Uses system default

}, {

‘bucket’ : ‘example-bucket-1’, ‘key’ : ‘example/asset2.jpg’, # if unspecified, auth uses localhost and system defaults

}, {

‘bucket’ : ‘example-bucket-2’, ‘key’ : ‘example/asset1-in-bucket-2.tif’, ‘auth_domain’ : ‘s3.amazon.com’, ‘auth_access_id’ : ‘____________________’, ‘auth_secret_key’ : ‘________________________________________’,

},

]

Note that you cannot specify AWS authentication access ids or secret keys using string uri’s. For specific authentication methods, please use the latter list of dictionaries.
image_uuid_list (list of str) – list of image UUIDs to be used in IBEIS IA
image_width_list (list of int) – list of image widths
image_height_list (list of int) – list of image heights
image_orig_name_list (list of str) – list of original image names
image_ext_list (list of str) – list of original image names
image_time_posix_list (list of int) – list of image’s POSIX timestamps
image_gps_lat_list (list of float) – list of image’s GPS latitude values
image_gps_lon_list (list of float) – list of image’s GPS longitude values
image_orientation_list (list of int) – list of image’s orientation flags
image_notes_list (list of str) – optional list of any related notes with the images
**kwargs – key-value pairs passed to the ibs.add_images() function.

CommandLine:

python -m ibeis.web.app --test-add_images_json

Example

>>> # WEB_DOCTEST
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> import ibeis
>>> web_instance = ibeis.opendb(db='testdb1')
>>> _payload = {
>>>     'image_uri_list': [
>>>         'https://upload.wikimedia.org/wikipedia/commons/4/49/Zebra_running_Ngorongoro.jpg',
>>>         {
>>>             'bucket'          : 'test-asset-store',
>>>             'key'             : 'caribwhale/20130903-JAC-0002.JPG',
>>>         },
>>>     ],
>>>     'image_uuid_list': [
>>>         uuid.UUID('7fea8101-7dec-44e3-bf5d-b8287fd231e2'),
>>>         uuid.UUID('c081119a-e08e-4863-a710-3210171d27d6'),
>>>     ],
>>>     'image_width_list': [
>>>         1992,
>>>         1194,
>>>     ],
>>>     'image_height_list': [
>>>         1328,
>>>         401,
>>>     ],
>>> }
>>> gid_list = ibeis.web.app.add_images_json(web_instance, **_payload)
>>> print(gid_list)
>>> print(web_instance.get_image_uuids(gid_list))
>>> print(web_instance.get_image_uris(gid_list))
>>> print(web_instance.get_image_paths(gid_list))
>>> print(web_instance.get_image_uris_original(gid_list))

ibeis.web.apis_json.delete_annots_json(ibs, annot_uuid_list)[source]¶

REST:: Method: POST URL: /api/annot/json/

Parameters:	annot_uuid_list (list of str) – list of annot UUIDs to be delete from IBEIS

ibeis.web.apis_json.delete_images_json(ibs, image_uuid_list)[source]¶

REST:: Method: POST URL: /api/image/json/

Parameters:	image_uuid_list (list of str) – list of image UUIDs to be delete from IBEIS

ibeis.web.apis_json.get_annot_age_months_est_max_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_age_months_est_max_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_age_months_est_min_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_age_months_est_min_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_bboxes_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_detect_confidence_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_exemplar_flags_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_image_gps_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_image_names_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_image_paths_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_image_unixtimes_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_image_uuids_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_name_rowids_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_name_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_notes_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_num_verts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_qualities_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_quality_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_rotated_verts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_sex_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_sex_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_species_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_species_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_thetas_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_verts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_yaw_texts_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_annot_yaws_json(ibs, annot_uuid_list, **kwargs)[source]¶

ibeis.web.apis_json.get_image_annot_uuids_json(ibs, image_uuid_list)[source]¶

ibeis.web.apis_json.get_image_uris_original_json(ibs, image_uuid_list)[source]¶

ibeis.web.apis_json.get_imageset_annot_aids_json(ibs, imageset_uuid_list)[source]¶

ibeis.web.apis_json.get_imageset_annot_uuids_json(ibs, imageset_uuid_list)[source]¶

ibeis.web.apis_json.get_valid_annot_uuids_json(ibs, **kwargs)[source]¶

ibeis.web.apis_json.get_valid_image_uuids_json(ibs, **kwargs)[source]¶

ibeis.web.apis_json.get_valid_imageset_uuids_json(ibs, **kwargs)[source]¶

ibeis.web.apis_json.get_valid_name_uuids_json(ibs, **kwargs)[source]¶

ibeis.web.apis_json.set_exemplars_from_quality_and_viewpoint_json(ibs, annot_uuid_list, annot_name_list, **kwargs)[source]¶

ibeis.web.apis_query module¶

Dependencies: flask, tornado

ibeis.web.apis_query.get_recognition_query_aids(ibs, is_known, species=None)[source]¶

DEPCIRATE

RESTful:: Method: GET URL: /api/query/recognition_query_aids/

ibeis.web.apis_query.make_review_image(aid, cm, qreq_, view_orientation=u'vertical', draw_matches=True)[source]¶

” Create the review image for a pair of annotations

CommandLine:

python -m ibeis.web.apis_query make_review_image --show

Example

>>> # SCRIPT
>>> from ibeis.web.apis_query import *  # NOQA
>>> import ibeis
>>> cm, qreq_ = ibeis.testdata_cm('PZ_MTEST', a='default:dindex=0:10,qindex=0:1')
>>> aid = cm.get_top_aids()[0]
>>> tt = ut.tic('make image')
>>> image = make_review_image(aid, cm, qreq_)
>>> ut.toc(tt)
>>> ut.quit_if_noshow()
>>> print('image.shape = %r' % (image.shape,))
>>> print('image.dtype = %r' % (image.dtype,))
>>> ut.print_object_size(image)
>>> import plottool as pt
>>> pt.imshow(image)
>>> ut.show_if_requested()

ibeis.web.apis_query.process_graph_match_html(ibs, **kwargs)[source]¶

RESTful:: Method: POST URL: /api/review/query/graph/

ibeis.web.apis_query.query_chips(ibs, qaid_list=None, daid_list=None, cfgdict=None, use_cache=None, use_bigcache=None, qreq_=None, return_request=False, verbose=False, save_qcache=None, prog_hook=None, return_cm_dict=False, return_cm_simple_dict=False)[source]¶

Submits a query request to the hotspotter recognition pipeline. Returns a list of QueryResult objects.

Parameters:	qaid_list (list) – a list of annotation ids to be submitted as queries daid_list (list) – a list of annotation ids used as the database that will be searched cfgdict (dict) – dictionary of configuration options used to create a new QueryRequest if not already specified use_cache (bool) – turns on/off chip match cache (default: True) use_bigcache (bool) – turns one/off chunked chip match cache (default: True) qreq (QueryRequest) – optional, a QueryRequest object that overrides all previous settings return_request (bool) – returns the request which will be created if one is not already specified verbose (bool) – default=False, turns on verbose printing
Returns:	a list of ChipMatch objects containing the matching annotations, scores, and feature matches
Return type:	list

Returns(2):

tuple: (cm_list, qreq_) - a list of query results and optionally the: QueryRequest object used

RESTful:

Method: PUT URL: /api/query/chips/

CommandLine:

python -m ibeis.web.apis_query --test-query_chips

# Test speed of single query
python -m ibeis --tf IBEISController.query_chips --db PZ_Master1 \
    -a default:qindex=0:1,dindex=0:500 --nocache-hs

python -m ibeis --tf IBEISController.query_chips --db PZ_Master1 \
    -a default:qindex=0:1,dindex=0:3000 --nocache-hs

python -m ibeis.web.apis_query --test-query_chips:1 --show
python -m ibeis.web.apis_query --test-query_chips:2 --show

Example

>>> # SLOW_DOCTEST
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> import ibeis
>>> qreq_ = ibeis.testdata_qreq_()
>>> ibs = qreq_.ibs
>>> cm_list = qreq_.execute()
>>> cm = cm_list[0]
>>> ut.quit_if_noshow()
>>> cm.ishow_analysis(qreq_)
>>> ut.show_if_requested()

Example

>>> # SLOW_DOCTEST
>>> #from ibeis.all_imports import *  # NOQA
>>> import ibeis
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> qaid_list = [1]
>>> daid_list = [1, 2, 3, 4, 5]
>>> ibs = ibeis.test_main(db='testdb1')
>>> qreq_ = ibs.new_query_request(qaid_list, daid_list)
>>> cm = ibs.query_chips(qaid_list, daid_list, use_cache=False, qreq_=qreq_)[0]
>>> ut.quit_if_noshow()
>>> cm.ishow_analysis(qreq_)
>>> ut.show_if_requested()

Example1:

>>> # SLOW_DOCTEST
>>> #from ibeis.all_imports import *  # NOQA
>>> import ibeis
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> qaid_list = [1]
>>> daid_list = [1, 2, 3, 4, 5]
>>> ibs = ibeis.test_main(db='testdb1')
>>> cfgdict = {'pipeline_root':'BC_DTW'}
>>> qreq_ = ibs.new_query_request(qaid_list, daid_list, cfgdict=cfgdict, verbose=True)
>>> cm = ibs.query_chips(qaid_list, daid_list, use_cache=False, qreq_=qreq_)[0]
>>> ut.quit_if_noshow()
>>> cm.ishow_analysis(qreq_)
>>> ut.show_if_requested()

ibeis.web.apis_query.query_chips_dict(ibs, *args, **kwargs)[source]¶

Runs query_chips, but returns a json compatible dictionary

RESTful:: Method: GET URL: /api/query/chips/dict/

ibeis.web.apis_query.query_chips_graph(ibs, qaid_list, daid_list, user_feedback=None, query_config_dict={}, echo_query_params=True)[source]¶

ibeis.web.apis_query.query_chips_simple_dict(ibs, *args, **kwargs)[source]¶

Runs query_chips, but returns a json compatible dictionary

Parameters:	as query_chips (same) –

RESTful:: Method: GET URL: /api/query/chips/simple_dict/
SeeAlso:: query_chips

CommandLine:

python -m ibeis.web.apis_query --test-query_chips_simple_dict:0
python -m ibeis.web.apis_query --test-query_chips_simple_dict:1

python -m ibeis.web.apis_query --test-query_chips_simple_dict:0 --humpbacks

Example

>>> # WEB_DOCTEST
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> #qaid = ibs.get_valid_aids()[0:3]
>>> qaids = ibs.get_valid_aids()
>>> daids = ibs.get_valid_aids()
>>> dict_list = ibs.query_chips_simple_dict(qaids, daids)
>>> qgids = ibs.get_annot_image_rowids(qaids)
>>> qnids = ibs.get_annot_name_rowids(qaids)
>>> for dict_, qgid, qnid in zip(dict_list, qgids, qnids):
>>>     dict_['qgid'] = qgid
>>>     dict_['qnid'] = qnid
>>>     dict_['dgid_list'] = ibs.get_annot_image_rowids(dict_['daid_list'])
>>>     dict_['dnid_list'] = ibs.get_annot_name_rowids(dict_['daid_list'])
>>>     dict_['dgname_list'] = ibs.get_image_gnames(dict_['dgid_list'])
>>>     dict_['qgname'] = ibs.get_image_gnames(dict_['qgid'])
>>> result  = ut.list_str(dict_list, nl=2, precision=2, hack_liststr=True)
>>> result = result.replace('u\'', '"').replace('\'', '"')
>>> print(result)

Example

>>> # WEB_DOCTEST
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> import time
>>> import ibeis
>>> import requests
>>> # Start up the web instance
>>> web_instance = ibeis.opendb_in_background(db='testdb1', web=True, browser=False)
>>> time.sleep(.5)
>>> baseurl = 'http://127.0.1.1:5000'
>>> data = dict(qaid_list=[1])
>>> resp = requests.get(baseurl + '/api/query/chips/simple_dict/', data=data)
>>> print(resp)
>>> web_instance.terminate()
>>> json_dict = resp.json()
>>> cmdict_list = json_dict['response']
>>> assert 'score_list' in cmdict_list[0]

ibeis.web.apis_query.query_chips_test(ibs, **kwargs)[source]¶

CommandLine:

python -m ibeis.web.apis_query query_chips_test

Example

>>> # SLOW_DOCTEST
>>> from ibeis.control.IBEISControl import *  # NOQA
>>> import ibeis
>>> qreq_ = ibeis.testdata_qreq_(defaultdb='testdb1')
>>> ibs = qreq_.ibs
>>> result_dict = ibs.query_chips_test()
>>> print(result_dict)

ibeis.web.apis_query.review_graph_match_html(ibs, review_pair, cm_dict, query_config_dict, _internal_state, callback_url, callback_method=u'POST', view_orientation=u'vertical', include_jquery=False)[source]¶

Parameters:	ibs (ibeis.IBEISController) – image analysis api review_pair (dict) – pair of annot uuids cm_dict (dict) – query_config_dict (dict) – _internal_state – callback_url – callback_method (unicode) – (default = u’POST’) view_orientation (unicode) – (default = u’vertical’) include_jquery (bool) – (default = False)

CommandLine:

python -m ibeis.web.apis_query review_graph_match_html --show

ibeis --web
python -m ibeis.web.apis_query review_graph_match_html --show --domain=localhost

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.apis_query import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web('testdb1')  # , domain='http://52.33.105.88')
>>> aids = web_ibs.send_ibeis_request('/api/annot/', 'get')[0:2]
>>> uuid_list = web_ibs.send_ibeis_request('/api/annot/uuids/', type_='get', aid_list=aids)
>>> quuid_list = uuid_list[0:1]
>>> duuid_list = uuid_list
>>> query_config_dict = {
>>>    # 'pipeline_root' : 'BC_DTW'
>>> }
>>> data = dict(
>>>     query_annot_uuid_list=quuid_list, database_annot_uuid_list=duuid_list,
>>>     query_config_dict=query_config_dict,
>>> )
>>> jobid = web_ibs.send_ibeis_request('/api/engine/query/graph/', **data)
>>> print('jobid = %r' % (jobid,))
>>> status_response = web_ibs.wait_for_results(jobid)
>>> result_response = web_ibs.read_engine_results(jobid)
>>> inference_result = result_response['json_result']
>>> print('inference_result = %r' % (inference_result,))
>>> auuid2_cm = inference_result['cm_dict']
>>> quuid = quuid_list[0]
>>> class_dict = auuid2_cm[str(quuid)]
>>> # Get information in frontend
>>> #ibs = ibeis.opendb('testdb1')
>>> #cm = match_obj = ibeis.ChipMatch.from_dict(class_dict, ibs=ibs)
>>> #match_obj.print_rawinfostr()
>>> # Make the dictionary a bit more managable
>>> #match_obj.compress_top_feature_matches(num=2)
>>> #class_dict = match_obj.to_dict(ibs=ibs)
>>> cm_dict = class_dict
>>> # Package for review
>>> review_pair = {'annot_uuid_1': quuid, 'annot_uuid_2': duuid_list[1]}
>>> callback_method = u'POST'
>>> view_orientation = u'vertical'
>>> include_jquery = False
>>> kw = dict(
>>>     review_pair=review_pair,
>>>     cm_dict=cm_dict,
>>>     query_config_dict=query_config_dict,
>>>     _internal_state=None,
>>>     callback_url = None,
>>> )
>>> html_str = web_ibs.send_ibeis_request('/api/review/query/graph/', type_='get', **kw)
>>> web_ibs.terminate2()
>>> ut.quit_if_noshow()
>>> import plottool as pt
>>> ut.render_html(html_str)
>>> ut.show_if_requested()

Example2:

>>> # DISABLE_DOCTEST
>>> # This starts off using web to get information, but finishes the rest in python
>>> from ibeis.web.apis_query import *  # NOQA
>>> import ibeis
>>> ut.exec_funckw(review_graph_match_html, globals())
>>> web_ibs = ibeis.opendb_bg_web('testdb1')  # , domain='http://52.33.105.88')
>>> aids = web_ibs.send_ibeis_request('/api/annot/', 'get')[0:2]
>>> uuid_list = web_ibs.send_ibeis_request('/api/annot/uuids/', type_='get', aid_list=aids)
>>> quuid_list = uuid_list[0:1]
>>> duuid_list = uuid_list
>>> query_config_dict = {
>>>    # 'pipeline_root' : 'BC_DTW'
>>> }
>>> data = dict(
>>>     query_annot_uuid_list=quuid_list, database_annot_uuid_list=duuid_list,
>>>     query_config_dict=query_config_dict,
>>> )
>>> jobid = web_ibs.send_ibeis_request('/api/engine/query/graph/', **data)
>>> status_response = web_ibs.wait_for_results(jobid)
>>> result_response = web_ibs.read_engine_results(jobid)
>>> web_ibs.terminate2()
>>> # NOW WORK IN THE FRONTEND
>>> inference_result = result_response['json_result']
>>> auuid2_cm = inference_result['cm_dict']
>>> quuid = quuid_list[0]
>>> class_dict = auuid2_cm[str(quuid)]
>>> # Get information in frontend
>>> ibs = ibeis.opendb('testdb1')
>>> cm = ibeis.ChipMatch.from_dict(class_dict, ibs=ibs)
>>> cm.print_rawinfostr()
>>> # Make the dictionary a bit more managable
>>> cm.compress_top_feature_matches(num=1)
>>> cm.print_rawinfostr()
>>> class_dict = cm.to_dict(ibs=ibs)
>>> cm_dict = class_dict
>>> # Package for review ( CANT CALL DIRECTLY BECAUSE OF OUT OF CONTEXT )
>>> review_pair = {'annot_uuid_1': quuid, 'annot_uuid_2': duuid_list[1]}
>>> x = review_graph_match_html(ibs, review_pair, cm_dict,
>>>                             query_config_dict, _internal_state=None,
>>>                             callback_url=None)
>>> ut.quit_if_noshow()
>>> import plottool as pt
>>> ut.render_html(html_str)
>>> ut.show_if_requested()

ibeis.web.apis_query.review_query_chips_test()[source]¶

CommandLine:

python -m ibeis.web.apis_query review_query_chips_test --show

Example

>>> # SCRIPT
>>> import ibeis
>>> #web_ibs = ibeis.opendb_bg_web('testdb1')  # , domain='http://52.33.105.88')
>>> #import webbrowser
>>> #webbrowser.open(web_ibs.baseurl + '/test/review/query/chips/?__format__=True')
>>> # DISABLE_DOCTEST
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web(
>>>     browser=true, url_suffix='/test/review/query/chips/?__format__=true')

ibeis.web.app module¶

Dependencies: flask, tornado

ibeis.web.app.start_from_ibeis(ibs, port=None, browser=None, precache=None, url_suffix=None, start_job_queue=True)[source]¶

Parse command line options and start the server.

CommandLine:

python -m ibeis --db PZ_MTEST --web
python -m ibeis --db PZ_MTEST --web --browser

ibeis.web.app.start_tornado(ibs, port=None, browser=None, url_suffix=None)[source]¶: Initialize the web server

ibeis.web.app.start_web_annot_groupreview(ibs, aid_list)[source]¶

Parameters:	ibs (IBEISController) – ibeis controller object aid_list (list) – list of annotation rowids

CommandLine:

python -m ibeis.tag_funcs --exec-start_web_annot_groupreview --db PZ_Master1
python -m ibeis.tag_funcs --exec-start_web_annot_groupreview --db GZ_Master1
python -m ibeis.tag_funcs --exec-start_web_annot_groupreview --db GIRM_Master1

Example

>>> # SCRIPT
>>> from ibeis.tag_funcs import *  # NOQA
>>> import ibeis
>>> #ibs = ibeis.opendb(defaultdb='PZ_Master1')
>>> ibs = ibeis.opendb(defaultdb='GZ_Master1')
>>> #aid_list = ibs.get_valid_aids()
>>> # -----
>>> any_tags = ut.get_argval('--tags', type_=list, default=['Viewpoint'])
>>> min_num = ut.get_argval('--min_num', type_=int, default=1)
>>> prop = any_tags[0]
>>> filtered_annotmatch_rowids = filter_annotmatch_by_tags(ibs, None, any_tags=any_tags, min_num=min_num)
>>> aid1_list = (ibs.get_annotmatch_aid1(filtered_annotmatch_rowids))
>>> aid2_list = (ibs.get_annotmatch_aid2(filtered_annotmatch_rowids))
>>> aid_list = list(set(ut.flatten([aid2_list, aid1_list])))
>>> result = start_web_annot_groupreview(ibs, aid_list)
>>> print(result)

ibeis.web.app.test_html_error()[source]¶

This test will show what our current errors look like

CommandLine:

python -m ibeis.web.app --exec-test_html_error

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.app import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web(browser=True, start_job_queue=False, url_suffix='/api/image/imagesettext/?__format__=True')

ibeis.web.appfuncs module¶

class ibeis.web.appfuncs.NavbarClass(nav)[source]¶: Bases: object

ibeis.web.appfuncs.check_valid_function_name(string)[source]¶

ibeis.web.appfuncs.convert_nmea_to_json(nmea_str, filename, GMT_OFFSET=0)[source]¶

ibeis.web.appfuncs.convert_old_viewpoint_to_yaw(view_theta)[source]¶

we initially had viewpoint coordinates inverted

Example

>>> import math
>>> TAU = 2 * math.pi
>>> old_viewpoint_labels = [
>>>     ('left'       ,   0, 0.000 * TAU,),
>>>     ('frontleft'  ,  45, 0.125 * TAU,),
>>>     ('front'      ,  90, 0.250 * TAU,),
>>>     ('frontright' , 135, 0.375 * TAU,),
>>>     ('right'      , 180, 0.500 * TAU,),
>>>     ('backright'  , 225, 0.625 * TAU,),
>>>     ('back'       , 270, 0.750 * TAU,),
>>>     ('backleft'   , 315, 0.875 * TAU,),
>>> ]
>>> fmtstr = 'old %15r %.2f -> new %15r %.2f'
>>> for lbl, theta, radians in old_viewpoint_labels:
>>>     print(fmtstr % (lbl, theta, lbl, convert_old_viewpoint_to_yaw(theta)))

ibeis.web.appfuncs.convert_yaw_to_old_viewpoint(yaw)[source]¶

we initially had viewpoint coordinates inverted

Example

>>> import math
>>> TAU = 2 * math.pi
>>> old_viewpoint_labels = [
>>>     ('left'       ,   0, 0.000 * TAU,),
>>>     ('frontleft'  ,  45, 0.125 * TAU,),
>>>     ('front'      ,  90, 0.250 * TAU,),
>>>     ('frontright' , 135, 0.375 * TAU,),
>>>     ('right'      , 180, 0.500 * TAU,),
>>>     ('backright'  , 225, 0.625 * TAU,),
>>>     ('back'       , 270, 0.750 * TAU,),
>>>     ('backleft'   , 315, 0.875 * TAU,),
>>> ]
>>> fmtstr = 'original_theta %15r %.2f -> yaw %15r %.2f -> reconstructed_theta %15r %.2f'
>>> for lbl, theta, radians in old_viewpoint_labels:
>>>     yaw = convert_old_viewpoint_to_yaw(theta)
>>>     reconstructed_theta = convert_yaw_to_old_viewpoint(yaw)
>>>     print(fmtstr % (lbl, theta, lbl, yaw, lbl, reconstructed_theta))

ibeis.web.appfuncs.decode_refer_url(encode)[source]¶

ibeis.web.appfuncs.default_species(ibs)[source]¶

ibeis.web.appfuncs.embed_image_html(imgBGR, target_width=1200.0)[source]¶: Creates an image embedded in HTML base64 format.

ibeis.web.appfuncs.encode_refer_url(url)[source]¶

ibeis.web.appfuncs.get_turk_annot_args(is_reviewed_func)[source]¶: Helper to return aids in an imageset or a group review

ibeis.web.appfuncs.imageset_annot_additional_processed(ibs, aid_list, nid_list)[source]¶

ibeis.web.appfuncs.imageset_annot_quality_processed(ibs, aid_list)[source]¶

ibeis.web.appfuncs.imageset_annot_viewpoint_processed(ibs, aid_list)[source]¶

ibeis.web.appfuncs.imageset_image_processed(ibs, gid_list)[source]¶

ibeis.web.appfuncs.movegroup_aid(ibs, aid, src_ag, dst_ag)[source]¶

ibeis.web.appfuncs.resize_via_web_parameters(image)[source]¶

ibeis.web.appfuncs.send_csv_file(string, filename)[source]¶

ibeis.web.appfuncs.template(template_directory=None, template_filename=None, **kwargs)[source]¶

ibeis.web.job_engine module¶

Accepts and handles requests for tasks.

Each of the following runs in its own Thread/Process.

BASICALLY DO A CLIENT/SERVER TO SPAWN PROCESSES AND THEN A PUBLISH SUBSCRIBE TO RETURN DATA

Accepter:

Receives tasks and requests Delegates tasks and responds to requests Tasks are delgated to an engine

Engine:

the engine accepts requests. the engine immediately responds WHERE it will be ready. the engine sends a message to the collector saying that something will be ready. the engine then executes a task. The engine is given direct access to the data.

Collector:

The collector accepts requests The collector can respond:

<ResultContent>

Results are ready.

Results are not ready.

Unknown jobid.

Error computing results.

Progress percent.

References

Simple task farm, with routed replies in pyzmq http://stackoverflow.com/questions/7809200/implementing-task-farm-messaging-pattern-with-zeromq https://gist.github.com/minrk/1358832

Notes

We are essentially goint to be spawning two processes. We can test these simultaniously using

python -m ibeis.web.job_engine test_job_engine

We can test these separately by first starting the background server

python -m ibeis.web.job_engine test_job_engine –bg

Alternative:: python -m ibeis.web.job_engine test_job_engine –bg –no-engine python -m ibeis.web.job_engine test_job_engine –bg –only-engine –fg-engine

And then running the forground process

python -m ibeis.web.job_engine test_job_engine –fg

class ibeis.web.job_engine.JobBackend[source]¶

Bases: object

initialize_background_processes(dbdir=None, wait=0)[source]¶

class ibeis.web.job_engine.JobInterface(jobiface, id_)[source]¶

Bases: object

get_job_result(jobiface, jobid)[source]¶

get_job_status(jobiface, jobid)[source]¶

get_unpacked_result(jobiface, jobid)[source]¶

initialize_client_thread(jobiface)[source]¶: Creates a ZMQ object in this thread. This talks to background processes.

queue_job(jobiface, action, callback_url=None, callback_method=None, *args, **kwargs)[source]¶

IBEIS:: This is just a function that lives in the main thread and ships off a job.
FIXME: I do not like having callback_url and callback_method specified: like this with args and kwargs. If these must be there then they should be specified first, or THE PREFERED OPTION IS args and kwargs should not be specified without the * syntax

The client - sends messages, and receives replies after they have been processed by the

wait_for_job_result(jobiface, jobid, timeout=10, freq=0.1)[source]¶

ibeis.web.job_engine.close_job_manager(ibs)[source]¶

ibeis.web.job_engine.collect_queue_loop()¶

ibeis.web.job_engine.collector_loop(dbdir)[source]¶: Service that stores completed algorithm results

ibeis.web.job_engine.engine_loop(id_, dbdir=None)[source]¶

IBEIS:

This will be part of a worker process with its own IBEISController instance.

Needs to send where the results will go and then publish the results there.

The engine_loop - receives messages, performs some action, and sends a reply, preserving the leading two message parts as routing identities

ibeis.web.job_engine.engine_queue_loop()[source]¶: Specialized queue loop

ibeis.web.job_engine.get_job_result(ibs, jobid)[source]¶: Web call that returns the result of a job

ibeis.web.job_engine.get_job_status(ibs, jobid)[source]¶

Web call that returns the status of a job

CommandLine:

# Run Everything together
python -m ibeis.web.job_engine --exec-get_job_status

# Start job queue in its own process
python -m ibeis.web.job_engine test_job_engine --bg
# Start web server in its own process
./main.py --web --fg
pass
# Run foreground process
python -m ibeis.web.job_engine --exec-get_job_status:0 --fg

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.job_engine import *  # NOQA
>>> import ibeis
>>> web_ibs = ibeis.opendb_bg_web('testdb1')  # , domain='http://52.33.105.88')
>>> # Test get status of a job id that does not exist
>>> response = web_ibs.send_ibeis_request('/api/engine/job/status/', jobid='badjob')
>>> web_ibs.terminate2()

ibeis.web.job_engine.initialize_job_manager(ibs)[source]¶

Starts a background zmq job engine. Initializes a zmq object in this thread that can talk to the background processes.

Run from the webserver

CommandLine:

python -m ibeis.web.job_engine --exec-initialize_job_manager:0

Example

>>> # DISABLE_DOCTEST
>>> from ibeis.web.job_engine import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> from ibeis.web import apis_engine
>>> from ibeis.web import job_engine
>>> ibs.load_plugin_module(job_engine)
>>> ibs.load_plugin_module(apis_engine)
>>> ibs.initialize_job_manager()
>>> print('Initializqation success. Now closing')
>>> ibs.close_job_manager()
>>> print('Closing success.')

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.job_engine import *  # NOQA
>>> import ibeis
>>> import requests
>>> web_instance = ibeis.opendb_bg_web(db='testdb1')
>>> baseurl = 'http://127.0.1.1:5000'
>>> _payload = {'image_attrs_list': [], 'annot_attrs_list': []}
>>> payload = ut.map_dict_vals(ut.to_json, _payload)
>>> resp1 = requests.post(baseurl + '/api/test/helloworld/?f=b', data=payload)
>>> #resp2 = requests.post(baseurl + '/api/image/json/', data=payload)
>>> #print(resp2)
>>> web_instance.terminate()
>>> #json_dict = resp2.json()
>>> #text = json_dict['response']
>>> #print(text)

ibeis.web.job_engine.make_queue_loop(iface1, iface2, name=None)[source]¶

Standard queue loop

Parameters:	iface1 (str) – address for the client that deals iface2 (str) – address for the server that routes name (None) – (default = None)

ibeis.web.job_engine.on_collect_request(collect_request, collecter_data, awaiting_data, shelve_path)[source]¶: Run whenever the collector recieves a message

ibeis.web.job_engine.on_engine_request(ibs, jobid, action, args, kwargs)[source]¶: Run whenever the engine recieves a message

ibeis.web.job_engine.rcv_multipart_json(sock, num=2, print=<function print>)[source]¶: helper

ibeis.web.job_engine.send_multipart_json(sock, idents, reply)[source]¶: helper

ibeis.web.job_engine.test_job_engine()[source]¶

CommandLine:

python -m ibeis.web.job_engine --exec-test_job_engine
python -b -m ibeis.web.job_engine --exec-test_job_engine

python -m ibeis.web.job_engine test_job_engine
python -m ibeis.web.job_engine test_job_engine --bg
python -m ibeis.web.job_engine test_job_engine --fg

Example

>>> # SCRIPT
>>> from ibeis.web.job_engine import *  # NOQA
>>> test_job_engine()

ibeis.web.job_engine.update_proctitle(procname)[source]¶

ibeis.web.job_engine.wait_for_job_result(ibs, jobid, timeout=10, freq=0.1)[source]¶

ibeis.web.routes module¶

Dependencies: flask, tornado

ibeis.web.routes.api_root()[source]¶

ibeis.web.routes.dbinfo()[source]¶

ibeis.web.routes.error404(exception=None)[source]¶

ibeis.web.routes.group_review()[source]¶

ibeis.web.routes.image_view_api(gid=None, thumbnail=False, fresh=False, **kwargs)[source]¶

Returns the base64 encoded image of image <gid>

RESTful:: Method: GET URL: /image/view/<gid>/

ibeis.web.routes.root()[source]¶

ibeis.web.routes.sightings(html_encode=True)[source]¶

ibeis.web.routes.turk()[source]¶

ibeis.web.routes.turk_additional()[source]¶

ibeis.web.routes.turk_detection()[source]¶

ibeis.web.routes.turk_detection_dynamic()[source]¶

ibeis.web.routes.turk_quality()[source]¶

PZ Needs Tags:: 17242 14468 14427 15946 14771 14084 4102 6074 3409

GZ Needs Tags; 1302

CommandLine:

python -m ibeis.web.app --exec-turk_quality --db PZ_Master1
python -m ibeis.web.app --exec-turk_quality --db GZ_Master1
python -m ibeis.web.app --exec-turk_quality --db GIRM_Master1

Example

>>> # SCRIPT
>>> from ibeis.other.ibsfuncs import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='testdb1')
>>> aid_list_ = ibs.find_unlabeled_name_members(qual=True)
>>> valid_views = ['primary', 'primary1', 'primary-1']
>>> aid_list = ibs.filter_aids_to_viewpoint(aid_list_, valid_views, unknown_ok=False)
>>> ibs.start_web_annot_groupreview(aid_list)

ibeis.web.routes.turk_viewpoint()[source]¶

CommandLine:

python -m ibeis.web.app --exec-turk_viewpoint --db PZ_Master1

Example

>>> # SCRIPT
>>> from ibeis.other.ibsfuncs import *  # NOQA
>>> import ibeis
>>> ibs = ibeis.opendb(defaultdb='PZ_Master1')
>>> aid_list_ = ibs.find_unlabeled_name_members(suspect_yaws=True)
>>> aid_list = ibs.filter_aids_to_quality(aid_list_, 'good', unknown_ok=False)
>>> ibs.start_web_annot_groupreview(aid_list)

ibeis.web.routes.upload()[source]¶

ibeis.web.routes.view()[source]¶

ibeis.web.routes.view_annotations()[source]¶

ibeis.web.routes.view_images()[source]¶

ibeis.web.routes.view_imagesets()[source]¶

ibeis.web.routes.view_names()[source]¶

ibeis.web.routes_ajax module¶

Dependencies: flask, tornado

ibeis.web.routes_ajax.annotation_src(aid=None)[source]¶

ibeis.web.routes_ajax.image_src(gid=None, thumbnail=False, fresh=False, **kwargs)[source]¶

ibeis.web.routes_ajax.set_cookie()[source]¶

ibeis.web.routes_csv module¶

Dependencies: flask, tornado

ibeis.web.routes_csv.download_sightings()[source]¶

ibeis.web.routes_csv.get_aid_list_csv()[source]¶

ibeis.web.routes_csv.get_demographic_info()[source]¶

ibeis.web.routes_csv.get_gid_list_csv()[source]¶

ibeis.web.routes_csv.get_gid_with_aids_csv()[source]¶

ibeis.web.routes_csv.get_image_info()[source]¶

ibeis.web.routes_csv.get_nid_with_gids_csv()[source]¶

ibeis.web.routes_submit module¶

Dependencies: flask, tornado

ibeis.web.routes_submit.group_review_submit()[source]¶

CommandLine:

python -m ibeis.web.app --exec-group_review_submit

Example

>>> # UNSTABLE_DOCTEST
>>> from ibeis.web.app import *  # NOQA
>>> import ibeis
>>> import ibeis.web
>>> ibs = ibeis.opendb('testdb1')
>>> aid_list = ibs.get_valid_aids()[::2]
>>> ibs.start_web_annot_groupreview(aid_list)

ibeis.web.routes_submit.submit_additional()[source]¶

ibeis.web.routes_submit.submit_detection()[source]¶

ibeis.web.routes_submit.submit_quality()[source]¶

ibeis.web.routes_submit.submit_viewpoint()[source]¶

ibeis.web.test_api module¶

This is a proof of concept for connecting to an authenticated Qubica Server

ibeis.web.test_api.get_api_result(uri, user_email=None, user_enc_pass=None, **kwargs)[source]¶: Make a GET API request to the server

ibeis.web.test_api.get_authorization_header(uri, user_email=None, user_enc_pass=None)[source]¶

ibeis.web.test_api.get_signature(key, message)[source]¶

ibeis.web.test_api.post_api_result(uri, user_email=None, user_enc_pass=None, **kwargs)[source]¶: Make a GET API request to the server

ibeis.web.test_api.run_test_api()[source]¶

CommandLine:

python -m ibeis.web.test_api --test-run_test_api

Example

>>> # WEB_DOCTEST
>>> from ibeis.web.test_api import *  # NOQA
>>> response = run_test_api()
>>> print('Server response: %r' % (response, ))
>>> result = response
(200, u'{"status": {"cache": -1, "message": "", "code": 200, "success": true}, "response": "testdb1"}', <bound method Response.json of <Response [200]>>)

ibeis.web package¶

Submodules¶

ibeis.web.apis module¶

ibeis.web.apis_detect module¶

ibeis.web.apis_engine module¶

ibeis.web.apis_json module¶

ibeis.web.apis_query module¶

ibeis.web.app module¶

ibeis.web.appfuncs module¶

ibeis.web.job_engine module¶

ibeis.web.routes module¶

ibeis.web.routes_ajax module¶

ibeis.web.routes_csv module¶

ibeis.web.routes_submit module¶

ibeis.web.test_api module¶

Module contents¶