java - Run Hadoop MapReduce Job from Dynamic Web Project on WildFly Servlet -


i have installed hadoop 2.2.0 on ubuntu 13.10 in pseudo-distributed mode in pc starts , runs correctly (only 1 datanode). i'm using eclipse kepler v4.3 maven plugin develop hadoop program , dynamic web project (web project without maven).

i have hadoop project called "hadwork" created hadwork.jar (right click on project, export, runnable jar file, option "extract required libraries generated jar"). works correctly when run job command line with: "hadoop jar hadwork.jar parameter1 parameter2 parameter3" , see correctly work progress on terminal.

now want run job dynamic web application deployed on wildfly 8.0 application server (eclipse configured run project on correct server wildfly in standalone mode). i'm writing servlet "servlethadooptest.java" don't understand how run job web application, libraries (hadoop libraries jar? hadwork jar?) need load , load it. can't use command "hadoop jar hadwork.jar parameter1 parameter2 parameter3" in servlet directly. in short, want: when click on button "startjob" on page "index.jsp", job hadwork begins work , shows "working job.." message in web page, possibly showing url track job. call mapreduce job servlet. can load hadwork.jar on application server (if so, where?).

i'm looking informations on google have not found answers yet..

there 2 options want do:

1) wrap command line call hadoop within servlet. that's super ugly easiest set up, not have mix web application code base , hadoop. here's example on how that:

    string[] cmd = new string[] { <hadoop command goes here> };     process process = runtime.getruntime().exec(cmd);

2) other, better alternative package hadoop code , hadoop dependencies in servlet. stringly suggest use maven dependecy management. have following run map reduce job servlet: - package hadoop , job jar - create configuration object reflects cluster (especially dfs , mapred hosts) - implement tool


Comments

Post a Comment

Popular posts from this blog

android - Get AccessToken using signpost OAuth without opening a browser (Two legged Oauth) -

i using signpost oauth access data magento server. have read various tutorials on same , reach point open browser user can enter credentials. however, per requirement have automate part. hence, user should not browser page. have set-up done on server side ( magento ), hit url , returned call page. same through program in android. i have tried below, consumer = new commonshttpoauthconsumer(key, secret); provider = new commonshttpoauthprovider(oauth_init_url,access_token_url, authorize_url); try { provider.retrieverequesttoken(consumer, oauth.out_of_band); log.d("tokens" , consumer.gettoken() + " -- " + consumer.gettokensecret()); } and request tokens. dont know how bypass next step. tried directly accessing accesstoken (stupid of me) provider.retrieveaccesstoken(consumer, "mycallback://callback"); no luck, ends in oauth.signpost.exception.oauthnotauthorizedexception: authorization failed (server replied 401). ...
Read more

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: mockito -

getting error org.mockito.exceptions.misusing.invaliduseofmatchersexception: invalid use of argument matchers! 1 matchers expected, 2 recorded. exception may occur if matchers combined raw values: //incorrect: somemethod(anyobject(), "raw string"); when using matchers, arguments have provided matchers. example: //correct: somemethod(anyobject(), eq("string matcher")); at when( getprogramservice .callservice(any(getprogramtype.class))) .thenreturn(jaxbresponse); please explain error , possible resolution the code posted looks fine; don't see problem it. code before or above causing exception. key here: invalid use of argument matchers! 1 matchers expected, 2 recorded. any , used above, doesn't return "special kind of null" or "special kind of getprogramtype"; don't exist. instead, returns null , side effect puts matcher on stack. whe...
Read more

google shop client API returns 400 bad request error while adding an item -

i'm working on solution add items in google shopping market java application. following tutorial real mess because of missing classes of laste release, after setting 1.5-beta maven managed make base example typecheck correctly classes. still can make simple task of adding 1 item. instead of posting whole code i'm posting part creates product feed. private product createproduct(string id) { product product = new product(); product.title = "red wool sweater"; product.content = new content( "text", "comfortable , soft, sweater keep warm on " + "cold winter nights. red , blue stripes."); product.appcontrol = new appcontrol(); product.appcontrol.addrequireddestination("productads"); product.externalid = id; product.lang = "it"; product.country = "it"; product.condition = "nuovo"; product.price = new price("eu...
Read more