java - Labeled Latent Dirichlet Allocation input values -


i doing tag prediction , keyword extraction on stackexchange posts. have ~36,000 posts consisting of title, body , tags. processes them filtering out noisy elements. after perform labeled latent dirichlet allocation (llda) obtained here.

when looking @ output, majority of first half of topic-keyword assignment pretty good, example:

topic 0: hardware  hardware 0.01417490938078998  apple  0.007714736647543383  macbook    0.004179344296774437  mac    0.003794235182959134  topic 1: mac  mac    0.09533364420104305  os 0.02075003721054881  mini   0.00682593613383348  macs   0.00435445224274711  topic 2: powerpc  powerpc    0.010548590021130589  ppc    0.007893573342376935  mac    0.0039821054483700795  ibook  0.003731934198917873  os 0.003471650527888505

however, more come close end of output file, topic-keyword assignments weird:

topic 976: shopping-recommendation difference  7.5409094336777e-5 intel   7.5409094336777e-5 ppc 7.5409094336777e-5 turn    7.5409094336777e-5  topic 977: pci-card difference  7.5409094336777e-5 intel   7.5409094336777e-5 ppc 7.5409094336777e-5 turn    7.5409094336777e-5  topic 978: tmux difference  7.5409094336777e-5 intel   7.5409094336777e-5 ppc 7.5409094336777e-5 turn    7.5409094336777e-5  topic 979: difference  7.5409094336777e-5 intel   7.5409094336777e-5 ppc 7.5409094336777e-5 turn    7.5409094336777e-5

can please explain why such wrong assignments in end? , also, why values extremely low?

as said before have ~36,000 posts, these values perform llda:

option.est = true; option.alpha = 50/920 // 920 number of topics option.beta = 0.1; option.niters = 3000; option.twords = 15; option.nburnin = 350; option.samplinglag = 256;

i found little no documentation previous values, trial , error found these fit best of have managed get. however, maybe better understanding can explain me and/or suggest values best?


Comments

Post a Comment

Popular posts from this blog

android - Get AccessToken using signpost OAuth without opening a browser (Two legged Oauth) -

i using signpost oauth access data magento server. have read various tutorials on same , reach point open browser user can enter credentials. however, per requirement have automate part. hence, user should not browser page. have set-up done on server side ( magento ), hit url , returned call page. same through program in android. i have tried below, consumer = new commonshttpoauthconsumer(key, secret); provider = new commonshttpoauthprovider(oauth_init_url,access_token_url, authorize_url); try { provider.retrieverequesttoken(consumer, oauth.out_of_band); log.d("tokens" , consumer.gettoken() + " -- " + consumer.gettokensecret()); } and request tokens. dont know how bypass next step. tried directly accessing accesstoken (stupid of me) provider.retrieveaccesstoken(consumer, "mycallback://callback"); no luck, ends in oauth.signpost.exception.oauthnotauthorizedexception: authorization failed (server replied 401). ...
Read more

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: mockito -

getting error org.mockito.exceptions.misusing.invaliduseofmatchersexception: invalid use of argument matchers! 1 matchers expected, 2 recorded. exception may occur if matchers combined raw values: //incorrect: somemethod(anyobject(), "raw string"); when using matchers, arguments have provided matchers. example: //correct: somemethod(anyobject(), eq("string matcher")); at when( getprogramservice .callservice(any(getprogramtype.class))) .thenreturn(jaxbresponse); please explain error , possible resolution the code posted looks fine; don't see problem it. code before or above causing exception. key here: invalid use of argument matchers! 1 matchers expected, 2 recorded. any , used above, doesn't return "special kind of null" or "special kind of getprogramtype"; don't exist. instead, returns null , side effect puts matcher on stack. whe...
Read more

google shop client API returns 400 bad request error while adding an item -

i'm working on solution add items in google shopping market java application. following tutorial real mess because of missing classes of laste release, after setting 1.5-beta maven managed make base example typecheck correctly classes. still can make simple task of adding 1 item. instead of posting whole code i'm posting part creates product feed. private product createproduct(string id) { product product = new product(); product.title = "red wool sweater"; product.content = new content( "text", "comfortable , soft, sweater keep warm on " + "cold winter nights. red , blue stripes."); product.appcontrol = new appcontrol(); product.appcontrol.addrequireddestination("productads"); product.externalid = id; product.lang = "it"; product.country = "it"; product.condition = "nuovo"; product.price = new price("eu...
Read more