cassandra - What does the method setRepeatLastToken do in Astyanax? -


i'm looking @ data reading recipes , examples in astyanax documentation. of them (i.e. query rows callback) include 

setrepeatlasttoken(false)

can explain used for? when should use it? looks defaults (true).

link javadoc: http://netflix.github.io/astyanax/javadoc/com/netflix/astyanax/query/allrowsquery.html#setrepeatlasttoken(boolean)

the source code com.netflix.astyanax.query.allrowsquery includes following comment:

 * there few important implementation details need considered.  * implementation assumes random partitioner used. consequently  * keyrange query done using tokens , not row keys. done because  * when using random partitioner tokens sorted while keys not.  * however, because multiple keys potentially map same token each  * incremental query cassandra repeat last token previous  * response. ensure no keys skipped. have  * important implications. first, last , potentially more (if  * have same token) row keys previous response repeat. second,  * if range of repeating tokens larger block size code  * enter infinite loop. can mitigated selecting block size  * large enough likelyhood of happening low.  * also, if application can tolerate potential skipped row keys  * call setrepeatlasttoken(false) turn off features.

i understand query done based on token range instead of key range. why rows potentially skipped if token wasn't repeated?

the source code comment pretty explains functionality of setrepeatlasttoken(boolean). here details:

according this post, cassandra uses md5 or murmurhash (depending on cassandra version) algorithm generating tokens keys. both of these algorithms fast can generate collisions (same token value different keys). because of that, there might multiple rows stored under same token (usually if data set large enough).

cassandra stores data on nodes based on tokens. when using random partitioner, data retrieval done in token order (not key order). makes sense because records read same node(s) in sequence , generate less traffic retrieving records random nodes in cluster.

when reading cassandra using astyanax paging, page (block) size may correspond middle of set of rows same token. when request next page comes, astyanax needs know whether start next token (an possibly miss rest of rows same token didn't fit last page) or repeat last token make sure rows last key read (but repeating 1 or possibly more rows previous page).

the code comment warns if page size small enough rows same token fit it, code may enter infinite loop if setrepeatlasttoken set true.

i hope helps else might wondering feature.


Comments

Post a Comment

Popular posts from this blog

android - Get AccessToken using signpost OAuth without opening a browser (Two legged Oauth) -

i using signpost oauth access data magento server. have read various tutorials on same , reach point open browser user can enter credentials. however, per requirement have automate part. hence, user should not browser page. have set-up done on server side ( magento ), hit url , returned call page. same through program in android. i have tried below, consumer = new commonshttpoauthconsumer(key, secret); provider = new commonshttpoauthprovider(oauth_init_url,access_token_url, authorize_url); try { provider.retrieverequesttoken(consumer, oauth.out_of_band); log.d("tokens" , consumer.gettoken() + " -- " + consumer.gettokensecret()); } and request tokens. dont know how bypass next step. tried directly accessing accesstoken (stupid of me) provider.retrieveaccesstoken(consumer, "mycallback://callback"); no luck, ends in oauth.signpost.exception.oauthnotauthorizedexception: authorization failed (server replied 401). ...
Read more

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: mockito -

getting error org.mockito.exceptions.misusing.invaliduseofmatchersexception: invalid use of argument matchers! 1 matchers expected, 2 recorded. exception may occur if matchers combined raw values: //incorrect: somemethod(anyobject(), "raw string"); when using matchers, arguments have provided matchers. example: //correct: somemethod(anyobject(), eq("string matcher")); at when( getprogramservice .callservice(any(getprogramtype.class))) .thenreturn(jaxbresponse); please explain error , possible resolution the code posted looks fine; don't see problem it. code before or above causing exception. key here: invalid use of argument matchers! 1 matchers expected, 2 recorded. any , used above, doesn't return "special kind of null" or "special kind of getprogramtype"; don't exist. instead, returns null , side effect puts matcher on stack. whe...
Read more

google shop client API returns 400 bad request error while adding an item -

i'm working on solution add items in google shopping market java application. following tutorial real mess because of missing classes of laste release, after setting 1.5-beta maven managed make base example typecheck correctly classes. still can make simple task of adding 1 item. instead of posting whole code i'm posting part creates product feed. private product createproduct(string id) { product product = new product(); product.title = "red wool sweater"; product.content = new content( "text", "comfortable , soft, sweater keep warm on " + "cold winter nights. red , blue stripes."); product.appcontrol = new appcontrol(); product.appcontrol.addrequireddestination("productads"); product.externalid = id; product.lang = "it"; product.country = "it"; product.condition = "nuovo"; product.price = new price("eu...
Read more