if statement - Python eval and logical operators -

i have json "database" - it's python list of json objects:

[{'_id': 'transaction0', 'offer': {'from': 'merchant1', 'to': 'customer1', 'item': 'car', 'price': 1000, 'timestamp': 2}, 'accept': {'quantity': 1, 'address': '123 fake street', 'timestamp': 5}}, {'_id': 'transaction1', 'offer': {'from': 'merchant1', 'to': 'customer2', 'item': 'computer', 'price': 500, 'timestamp': 5}}, {'_id': 'transaction2', 'offer': {'from': 'merchant3', 'to': 'customer3', 'item': 'garbage bin', 'price': 10, 'timestamp': 0}, 'accept': {'quantity': 2, 'address': '456 madeup road', 'timestamp': 1}}, {'_id': 'transaction3', 'offer': {'from': 'merchant2', 'to': 'customer1', 'item': 'car', 'price': 2000, 'timestamp': 3}, 'accept': {'quantity': 2, 'address': 'the white house', 'timestamp': 3}}, {'_id': 'transaction4', 'offer': {'from': 'merchant3', 'to': 'customer3', 'item': 'pens', 'price': 2, 'timestamp': 0}, 'accept': {'quantity': 4, 'address': 'houses of parliment', 'timestamp': 1}}, {'_id': 'transaction5', 'offer': {'from': 'merchant4', 'to': 'customer1', 'item': 'headphones', 'price': 200, 'timestamp': 4}}, {'_id': 'transaction6', 'offer': {'from': 'merchant1', 'to': 'customer2', 'item': 'water bottle', 'price': 1, 'timestamp': 1}, 'accept': {'quantity': 3, 'address': 'timbuktu', 'timestamp': 14}}, {'_id': 'transaction7', 'offer': {'from': 'merchant2', 'to': 'customer3', 'item': 'laptop', 'price': 900, 'timestamp': 0}}, {'_id': 'transaction8', 'offer': {'from': 'merchant4', 'to': 'customer1', 'item': 'chair', 'price': 80, 'timestamp': 3}, 'accept': {'quantity': 1, 'address': 'mordor', 'timestamp': 3}}, {'_id': 'transaction9', 'offer': {'from': 'merchant3', 'to': 'customer3', 'item': 'garbage bin', 'price': 5, 'timestamp': 2}, 'accept': {'quantity': 2, 'address': 'the wall', 'timestamp': 2}}] 

my intention use queries, stored in dictionaries, against database. in example, dictionary contains:

a_dict = {"query1": "'offer' , 'accept'"} 

note dictionary contain more queries, , more complicated queries (e.g. (cond1 , cond2) or (cond2 , cond3)), need understand why python doing it's doing (and how overcome it) opposed solely solution is.

i need have evaluates , runs query1 correctly. wrongful implementation currently:

if (eval(a_dict["query1"]) + "in i"): 

this same as:

if 'offer' , 'accept' in i: 

due short-circuiting, evaluates checking whether accept in i. in example, everytime there accept there offer, may not case.

a rightful if statement be:

if 'offer' in , 'accept' in i: 

however, isn't composable type of potential queries have. ideally, i'd have elegant solution "plug , play", similar eval if statement given above.

is there anyway able take particular query dictionary, plug if statement, , run if statement i'm intending (under assumption queries make logical sense)?

https://www.python.org/dev/peps/pep-0308/ article says faq 4.16 gives alternatives, can't seem find anywhere

please don't use eval queries. guaranteed blow in face when don't expect it. maybe you've heard of sql injections; security implications of using eval build queries huge.

a filter-based query system

instead, begin writing filter functions common queries. solve problem , provide "plug-and-play" way compose queries.

here's pointer how implement it:

think of query function takes arguments few literal values (and, implicitly, set of records), , returns resultset of records. ditching list , using set datatype resultset, keyed record id, increase performance lot.

then "and" becomes function takes 2 (or more) sets of records , builds set intersection of them, , "or" becomes function takes 2 (or more) sets of records , builds union of them. (not set difference between whole set of records , 1 or more subsets).

if build functions way, query become simple tree of function calls, such as:

result = q_and(q_or(q_merchant_is('merchant2'),                      q_address_is('123 fakestreet')),                 q_quantity_above(3)) 

(formatted better legibility)

it's not hard write parser simple query language build such query, if don't need provide frontend endusers, might not need own query language because python representation of query seen above simple , clear enough. , if need represent queries dictionaries, well, if choose structure closely mimicks final structure of query call tree, it's trivial write query_builder function turns 1 of dict queries function run tree of query function calls when called.

note: can see, q_merchant_is, q_quantity_above etc don't take set of records filter. can fix making query class , set full set instance attribute, each query method has access full recordset if needs it:

class query(object):     def __init__(self, all_records):         self.records = all_records      def merchant_is(self, name):         result = set()         record in self.records:             if record['offer']['from'] == name:                result.add(record['_id'])         return result      def q_and(self, *args):         result = args[0]         in range(1, len(args)):             result = args[i].intersection(result)         return result     ...  q = query(my_full_record_set) result = q.q_and(q.q_or(q.merchant_is('merchant2').........))     

performance , indices

you see each query function queries literal value scans on whole dataset filter it. if query contains many such searches literal parts, you'll scan dataset multiple times. large datasets, can become prohibitive.

a simple solution index fields want query against in 1 dict per field. speed query orders of magnitude, if data changed, you'd need make sure keep indices date.

classifier query system

another solution build query functions classifiers instead of filters, meaning merchant_is take literal value , record , answer true or false, depending on whether record contained literal value in right field. make work efficiently having factory functions build composite query.

the example query filter section become:

query = q_and(q_or(q_merchant_is('merchant2'),                    q_address_is('123 fakestreet')),               q_quantity_above(3)) result = perform_query(query, all_my_records) 

q_merchant_is turn following:

def q_merchant_is(literal):     return lambda record: record['orders']['from'] == literal 

note how you're returning function that, when called record, classify it.

q_or might this:

def q_or(*args):     def or_template(record):         classifier in args:             if classifier(record):                 return true         return false     return or_template 

or bit terser (i'm not sure whether more efficient or not):

def q_or(*args):     return lambda record: any([ classifier(record) classifier in args]) 

q_or returns function runs number of classifiers against record passed argument , returns true if @ least 1 of classifiers returns true. q_and works q_or except returns true if every classifier returns true. , q_not return true if it's classifier returned false, , vice versa.

now need is:

def perform_query(query, all_records):     return filter(query, all_records) 

this iterate on dataset single time , pretty efficient can in python without involving eval, compile , exec, it's harder understand filter approach.

however, isn't composable type of potential queries have. ideally, i'd have elegant solution "plug , play"

with both filter , classifier systems, easy extend system new query elements. in filter example, add method query class. in classifier example, add query function builder 1 wrote q_merchant_is. involves 2 lines of python code.