how to load url asynchronusly in javafx browser?

[Niti Kapoor]

im making a crawler application to download news from all epapers
so i want to load the url asynchronously as after loading one url in window the task of downloading show complete than only second url should load which im getting from json.

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package mms;

import com.eclipsesource.json.Json;
import com.eclipsesource.json.JsonValue;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import javafx.scene.control.Button;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.StackPane;
import javafx.scene.layout.VBox;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

/**
 *
 * @author biznis
 */
public class HV1 extends Application {

    /**
     *
     * @param primaryStage
     * @throws Exception
     */
    @Override
    public void start(Stage primaryStage) throws Exception {
        StackPane root = new StackPane();
        HBox hbox = new HBox(30); // create a HBox to hold 2 vboxes        

        // create a vbox with a textarea that grows vertically
        VBox vbox = new VBox(10);
        //Label label1 = new Label("");
        final WebView img = new WebView();
        final WebEngine Img = img.getEngine();
        Img.load("http://mms.bulksms101.com/api/crawler/downloads");
        final WebView img1 = new WebView();
        final WebEngine Img1 = img.getEngine();
        Img1.load("http://mms.bulksms101.com/api/crawler/downloads");
        VBox vbox2 = new VBox();
        final WebView browser = new WebView();
        final WebEngine wb2 = browser.getEngine();
        URL oracle = new URL("http://mms.bulksms101.com/api/crawler/downloads");
        try (BufferedReader in = new BufferedReader(
                new InputStreamReader(oracle.openStream()))) {
            String inputLine;
            while ((inputLine = in.readLine()) != null) {
                System.out.println(inputLine);
                com.eclipsesource.json.JsonObject object = Json.parse(inputLine).asObject();
                com.eclipsesource.json.JsonArray items = Json.parse(inputLine).asObject().get("data").asArray();
                for (JsonValue item : items) {
                    String name = item.asObject().getString("id", "Unknown Item");
                    String quantity = item.asObject().getString("url", "un");
                    System.out.println(quantity);
                    wb2.load(quantity);
                }
            }
            VBox vb = new VBox();
            Button btn1 = new Button();
            btn1.setText("Load");
            vb.getChildren().add(btn1);
            Button btn2 = new Button();
            btn2.setText("Download");
            vb.getChildren().add(btn2);
            Button btn3 = new Button();
            btn3.setText("Save");
            vb.getChildren().add(btn3);
            vbox.getChildren().addAll(img, img1);
            vbox2.setPadding(new Insets(3));
            vbox2.setSpacing(3);
            vbox2.getChildren().addAll(browser, vb);

            HBox.setHgrow(vbox2, Priority.ALWAYS);

            hbox.setPadding(new Insets(20));
//        StackPane.setMargin(hbox, new Insets(20));

            hbox.getChildren().addAll(vbox, vbox2);
            root.getChildren().add(hbox);
            Scene scene = new Scene(root, 500, 300); // the stack pane is the root node
            //scene.setCursor(Cursor.CROSSHAIR);

            primaryStage.setScene(scene);
            primaryStage.show();
            //btn2.setOnAction(new EventHandler<ActionEvent>() {
            //  @Override public void handle(ActionEvent event) {
            // moves++;
            //  }
//});
        }
    }

    public static void main(String[] args) {
        Application.launch(args);

    }

}

this is my complete code of application
im loading source url in first window and second window im trying to load url that im getting from json
and in third window i have to save function after finishing load in second window so i have to load url one by one (asynchronous) so that till that the second url should wait till the first url complete its task.

[Tim Moores]

If it's a crawler, why do you want the app to have a GUI? The user will look at just a single web page at any given time anyway. If you want to load several pages simultaneously, you can create a CardLayout for each page, and embed webview components in every one. Just make sure they run in different threads.

[Niti Kapoor]

its a company requirement.

[Tim Moores]

OK. As a software engineer, you need to learn to push back in strange requirements, though. Anyway, you got your answer.

[Niti Kapoor]

ya i know thanks no i dont want to load url simultaneously. i want it asynchronously that the url which i decode from json should load one by one asone url complete it task than other should load.

[Niti Kapoor]

why i should create cardlayout for every url one url should load at a time and then after first url completes its work than only other should load.

[Tim Moores]

So when you said "asynchronously", you actually meant "synchronously"? Because that's what it means when you load one after the other.

Regarding CardLayout: I assumed that you wanted to display the web pages; not so? If not, then what is the purpose of using the JavaFX WebView?

[Niti Kapoor]

yes i want to display web pages only but one after another in second webview.

[Niti Kapoor]

yes sry synchronously

[Tim Moores]

So what is the actual problem you're facing? You say you want to have two webviews for two web pages. If you want to load them one after another, check out the WebEngine.getLoadWorker method- it allows you to get access to the class that loads the web page, so you can check its status, and start the 2nd download after the first one has finished.

[Niti Kapoor]

one webview only not two in webview when the task of first url completed load second and so on.

[Niti Kapoor]

{"data":[{"id":"0193a95d6be46103f23392ec07be55e3","domain":"","url":"",
"lastdone":"2017-12-13 07:21:25","nextdue":"0000-00-00 00:00:00"},
{"id":"043afba87756972b8e23b4cb6d9624ce","domain":"","url":"",
"lastdone":"2017-12-11 10:51:21","nextdue":"2018-12-11 10:51:22"},
{"id":"138a28186fff71a6997aab9fa2a1a355","domain":"",
"url":"http:\/\/www.bbc.com","lastdone":"2017-09-06 07:37:13","nextdue":"2017-09-06 07:37:13"},
{"id":"148a28186fff71a6997aab9fa2a1a355","domain":"",
"url":"http:\/\/www.cnn.com","lastdone":"0000-00-00 00:00:00","nextdue":"0000-00-00 00:00:00"},
{"id":"9c566b6ccae9b2c8109d4bf1e546db1e","domain":"", "url":"","lastdone":"2017-12-12 06:47:21", "nextdue":"2018-12-12 06:47:24"},
{"id":"a7e1aca3ec8e8b9d7385e88d427c3f38","domain":"","url":"","lastdone":"2017-12-12 04:25:31","nextdue":"2018-12-12 04:25:31"},
{"id":"a8cf8081c1c3190ae6dafec9ec5a1166","domain":"","url":"","lastdone":"2017-12-11 11:23:54","nextdue":"0000-00-00 00:00:00"}],
"errors":"","meta":{"db":{"calc":{"start":0,"num":30,"found_rows":"7","affected_rows":0}},
"session_id":"ff16a926e801c6a87132af546680480a"}}

this is json from which im extracting url and load url one by one in one webview.

[Tim Moores]

So no user will actually look at this, it will just load one web page after another? It makes no sense to use a GUI for this, then - what is the actual purpose of downloading the pages? Why is a non-GUI tool like HtmlUnit not better for this? Please give a technical reason, not something like "my manager told me so".

[John Damien Smith]

One reason something like WebView may be better than something like HtmlUnit is that WebView is a full browser environment, so it will execute any javascript or stuff related to html5 extensions to Java which something like HtmlUnit will not do.  If the page is loading stuff with Ajax or dynamically adding stuff to the HTML DOM using JavaScript, which is pretty common nowadays for a lot of sites, then WebView would correctly render this to create the DOM, but HtmlUnit would not.  If the purpose is to create something like a library of reasonably accurate screenshots of rendered pages, the WebView would be reasonable for that, but HtmlUnit would not.  However... I have no idea what Niti is really trying to do, so I couldn't say whether one approach or the other would be helpful for Niti or not...

[Niti Kapoor]

specifically i have to load the url one by one in webview than after that i will get the html of that page which i need to  post in other url i hope you understand yes john you are some what right.

[Niti Kapoor]

https://stackoverflow.com/questions/40738150/load-multiple-urls-in-webview-one-by-one i  had to do something like this.

[Tim Moores]

John Damien Smith wrote:If the page is loading stuff with Ajax or dynamically adding stuff to the HTML DOM using JavaScript, which is pretty common nowadays for a lot of sites, then WebView would correctly render this to create the DOM, but HtmlUnit would not.

Why wouldn't it do that? HtmlUnit supports quite a few JavaScript libraries, especially jQuery, and has no principal problems executing JavaScript.

Of course, if some content is loaded only after some user interaction, then automation with HtmlUnit would be trickier. But in that case, using the JavaFX WebView wouldn't help either (unless a user sits in front of it, and indicates to the app at which point the page is at a state where it should be saved). But I agree, it's bit hard to tell, as the problem statement is not clear on that point.

[John Damien Smith]

> Why wouldn't it do that? HtmlUnit supports quite a few JavaScript libraries, especially jQuery, and has no principal problems executing JavaScript.

I probably should have looked into HtmlUnit a bit more closely before posting earlier, as I didn't know that :-).  Based upon what you write, HtmlUnit would likely suffice for many tasks.  My guess (and it is only a guess) is that WebView might be a bit more functional and compatible  than HtmlUnit as far as implementation of html5 style features, because WebView is based upon a WebKit implementation which has likely undergone more extensive compatibility testing against the html5 specs and various websites than HtmlUnit.  But yes, both approaches would probably be fine depending upon the actual application required.  HtmlUnit would probably be preferred if you wanted to do evaluation and testing of the page load results because I think it was created to directly support those kind of scenarios.  WebView would be good if you want an actual rendering that you can snapshot to an image or something like that.

[Niti Kapoor]

its just like a web crawler.

[Niti Kapoor]

import com.eclipsesource.json.Json;
import com.eclipsesource.json.JsonValue;
import java.awt.event.ActionEvent;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.logging.Level;
import java.util.logging.Logger;
import javafx.scene.control.Button;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.event.EventHandler;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.StackPane;
import javafx.scene.layout.VBox;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

/**
 *
 * @author biznis
 */
public class HV1 extends Application {

    /**
     *
     * @param primaryStage
     * @throws Exception
     */
    @Override
    public void start(Stage primaryStage) throws Exception {
        StackPane root = new StackPane();
        HBox hbox = new HBox(30); // create a HBox to hold 2 vboxes        

        // create a vbox with a textarea that grows vertically
        VBox vbox = new VBox(10);
        //Label label1 = new Label("");
        final WebView img = new WebView();
        final WebEngine Img = img.getEngine();
        Img.load("http://mms.bulksms101.com/api/crawler/downloads");
        final WebView img1 = new WebView();
        final WebEngine Img1 = img.getEngine();
        Img1.load("http://mms.bulksms101.com/api/crawler/downloads");
        VBox vbox2 = new VBox();
        final WebView browser = new WebView();
        final WebEngine wb2 = browser.getEngine();
        URL oracle = new URL("http://mms.bulksms101.com/api/crawler/downloads");
        try (BufferedReader in = new BufferedReader(
                new InputStreamReader(oracle.openStream()))) {
            String inputLine;
            while ((inputLine = in.readLine()) != null) {
                System.out.println(inputLine);
                com.eclipsesource.json.JsonObject object = Json.parse(inputLine).asObject();
                com.eclipsesource.json.JsonArray items = Json.parse(inputLine).asObject().get("data").asArray();
            
                for (JsonValue item : items) {
                    //System.out.println(item.toString());
                    String name = item.asObject().getString("id", "Unknown Item");
                    System.out.println(name);
                    String quantity = item.asObject().getString("url", "id");
                     System.out.println(quantity);
               
                   /* Platform.runLater(() ->{
                        try {
                            Thread.sleep(10000);
                        } catch (InterruptedException ex) {
                            Logger.getLogger(HV1.class.getName()).log(Level.SEVERE, null, ex);
                        }*/
                    wb2.load(quantity);
              
        /*          try{  
       Thread.sleep(400);  
      }catch(InterruptedException e){System.out.println(e);}  */
                  
                }
                
                
            }
            
            VBox vb = new VBox();
            Button btn1 = new Button();
            btn1.setText("Load");
            vb.getChildren().add(btn1);
            Button btn2 = new Button();
            btn2.setText("Download");
            vb.getChildren().add(btn2);
            Button btn3 = new Button();
            btn3.setText("Save");
            vb.getChildren().add(btn3);
            vbox.getChildren().addAll(img, img1);
            vbox2.setPadding(new Insets(3));
            vbox2.setSpacing(3);
            vbox2.getChildren().addAll(browser, vb);

            HBox.setHgrow(vbox2, Priority.ALWAYS);

            hbox.setPadding(new Insets(20));
//        StackPane.setMargin(hbox, new Insets(20));

            hbox.getChildren().addAll(vbox, vbox2);
            root.getChildren().add(hbox);
            Scene scene = new Scene(root, 500, 300); // the stack pane is the root node
            //scene.setCursor(Cursor.CROSSHAIR);

            primaryStage.setScene(scene);
            primaryStage.show();
           /* btn2.setOnAction(new EventHandler<ActionEvent>() {
             @Override
             public void handle(ActionEvent event) {
                  URL url;
    InputStream is = null;
    BufferedReader br;
    String line;
             url = new URL(quantity);
        is = url.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
             }
             }
        });
    }*/
        }
    } 
    public static void main(String[] args) {
        Application.launch(args);
    }

}

im able to load the url but the last url in webview but i want toi load from starting one by one basically the flow is when the url is loaded than i will click on downloaded button and downloads each URL html
waits for download to complete

fires save function when download complete

save function
save_url=http://mms.bulksms101.com/api/crawler/server/ID

replaces ID in url from source
puts or posts
content=download body
meta=headers in json format
and gui we need is because everyday they want to work on that as url will changes everyday so  for rectifying its easy to view on gui hope this time you understand

[Niti Kapoor]

browser1 - url will load one by one browser 2 html will download and browser3- downloaded html will post to the url http://mms.bulksms101.com/api/crawler/server/ID
with id of the url in json

[Niti Kapoor]

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package mms;

import com.eclipsesource.json.Json;
import com.eclipsesource.json.JsonValue;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import javafx.scene.control.Button;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.StackPane;
import javafx.scene.layout.VBox;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

/**
 *
 * @author biznis
 */
public class HV1 extends Application {

    /**
     *
     * @param primaryStage
     * @throws Exception
     *///
    @Override
    public void start(Stage primaryStage) throws Exception {
        StackPane root = new StackPane();
        HBox hbox = new HBox(30); // create a HBox to hold 2 vboxes        

        // create a vbox with a textarea that grows vertically
        VBox vbox = new VBox(10);
        //Label label1 = new Label("");
        final WebView img = new WebView();
        final WebEngine Img = img.getEngine();
       // Img.load("http://mms.bulksms101.com/api/crawler/downloads");
        final WebView img1 = new WebView();
        final WebEngine Img1 = img.getEngine();
        Img1.load("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/downloads");
        VBox vbox2 = new VBox();
        final WebView browser = new WebView();
        final WebEngine wb2 = browser.getEngine();
        URL oracle = new URL("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/downloads");
        try (BufferedReader in = new BufferedReader(
                new InputStreamReader(oracle.openStream()))) {
            String inputLine1;
            while ((inputLine1 = in.readLine()) != null) {
                System.out.println(inputLine1);
                com.eclipsesource.json.JsonObject object = Json.parse(inputLine1).asObject();
                com.eclipsesource.json.JsonArray items = Json.parse(inputLine1).asObject().get("data").asArray();
            
                for (JsonValue item : items) {
                    //System.out.println(item.toString());
                    String name = item.asObject().getString("id", "Unknown Item");
                    System.out.println(name);
                   
                    String quantity = item.asObject().getString("url", "id");
                   // JSONArray jsonArray2 = new JSONArray(quantity);
                     System.out.println(quantity);
               
                   /* Platform.runLater(() ->{
                        try {
                            Thread.sleep(10000);
                        } catch (InterruptedException ex) {
                            Logger.getLogger(HV1.class.getName()).log(Level.SEVERE, null, ex);
                        }*/
                    Img.load(quantity);
                            URL url;
    InputStream is = null;
    BufferedReader br;
    String line;
             url = new URL(quantity);
        is = url.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
            byte[] postData= line.getBytes( StandardCharsets.UTF_8 );
            wb2.load(line);
            String originalUrl = "https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/server/ID";
String newUrl = originalUrl.replace("ID", name);
System.out.println(newUrl);

String request        = newUrl;
URL    url1            = new URL( request );
HttpURLConnection conn= (HttpURLConnection) url1.openConnection();           
conn.setDoOutput( true );
conn.setInstanceFollowRedirects( false );
conn.setRequestMethod( "POST" );
conn.setRequestProperty( "Content-Type", "text/plain"); 
conn.setRequestProperty( "charset", "utf-8");
//conn.setRequestProperty( "Content-Length", Integer.toString( line ));
conn.setUseCaches( false );
try( DataOutputStream wr = new DataOutputStream( conn.getOutputStream())) {
 wr.write(postData);
            
             }
     
        /*          try{  
       Thread.sleep(400);  
      }catch(InterruptedException e){System.out.println(e);}  */
                  
                }
                
                
            }
            
            VBox vb = new VBox();
            Button btn1 = new Button();
            btn1.setText("Load");
            vb.getChildren().add(btn1);
            Button btn2 = new Button();
            btn2.setText("Download");
            vb.getChildren().add(btn2);
            Button btn3 = new Button();
            btn3.setText("Save");
            vb.getChildren().add(btn3);
            vbox.getChildren().addAll(img, img1);
            vbox2.setPadding(new Insets(3));
            vbox2.setSpacing(3);
            vbox2.getChildren().addAll(browser, vb);

            HBox.setHgrow(vbox2, Priority.ALWAYS);

            hbox.setPadding(new Insets(20));


            hbox.getChildren().addAll(vbox, vbox2);
            root.getChildren().add(hbox);
            Scene scene = new Scene(root, 500, 300); // the stack pane is the root node
            

            primaryStage.setScene(scene);
            primaryStage.show();
         //  btn2.setOnAction(new EventHandler<ActionEvent>() {
            // @Override
           //  public void handle(ActionEvent event) {
                /*  URL url;
    InputStream is = null;
    BufferedReader br;
    String line;
             url = new URL(quantity);
        is = url.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
             }
             }
        });
    }*/
        }
    } 
    }
    public static void main(String[] args) {
        Application.launch(args);
    }
}

thecode is running but at end im getting exception

Exception in Application start method
java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:389)
	at com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:328)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
Caused by: java.lang.RuntimeException: Exception in Application start method
	at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:917)
	at com.sun.javafx.application.LauncherImpl.lambda$launchApplication$154(LauncherImpl.java:182)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.MalformedURLException: no protocol: 
	at java.net.URL.<init>(URL.java:593)
	at java.net.URL.<init>(URL.java:490)
	at java.net.URL.<init>(URL.java:439)
	at mms.HV1.start(HV1.java:86)
	at com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$161(LauncherImpl.java:863)
	at com.sun.javafx.application.PlatformImpl.lambda$runAndWait$174(PlatformImpl.java:326)
	at com.sun.javafx.application.PlatformImpl.lambda$null$172(PlatformImpl.java:295)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.sun.javafx.application.PlatformImpl.lambda$runLater$173(PlatformImpl.java:294)
	at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
	at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
	... 1 more
Exception running application mms.HV1
C:\Users\biznis\AppData\Local\NetBeans\Cache\8.2\executor-snippets\run.xml:53: Java returned: 1
BUILD FAILED (total time: 4 minutes 28 seconds)

how to send in key value pair as now im post in text/plain but i have send like in content the hml part and in meta  in meta part

[Dave Tolls]

url = new URL(quantity);

At this point the value of quanity is not a full URL.
Print it out to see what value it holds.

Indeed, looking at the code, you are printing it out on line 73, so just check your logs.

[Niti Kapoor]

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package mms;

import com.eclipsesource.json.Json;
import com.eclipsesource.json.JsonValue;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.LinkedHashMap;
import java.util.Map;
import javafx.scene.control.Button;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.StackPane;
import javafx.scene.layout.VBox;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

/**
 *
 * @author biznis
 */
public class HV1 extends Application {

    /**
     *
     * @param primaryStage
     * @throws Exception
     *///
    @Override
    public void start(Stage primaryStage) throws Exception {
        StackPane root = new StackPane();
        HBox hbox = new HBox(30); // create a HBox to hold 2 vboxes        

        // create a vbox with a textarea that grows vertically
        VBox vbox = new VBox(10);
        //Label label1 = new Label("");
        final WebView img = new WebView();
        final WebEngine Img = img.getEngine();
       // Img.load("http://mms.bulksms101.com/api/crawler/downloads");
        final WebView img1 = new WebView();
        final WebEngine Img1 = img.getEngine();
        Img1.load("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/downloads");
        VBox vbox2 = new VBox();
        final WebView browser = new WebView();
        final WebEngine wb2 = browser.getEngine();
        URL oracle = new URL("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/downloads");
        try (BufferedReader in = new BufferedReader(
                new InputStreamReader(oracle.openStream()))) {
            String inputLine1;
            while ((inputLine1 = in.readLine()) != null) {
                System.out.println(inputLine1);
                com.eclipsesource.json.JsonObject object = Json.parse(inputLine1).asObject();
                com.eclipsesource.json.JsonArray items = Json.parse(inputLine1).asObject().get("data").asArray();
            
                for (JsonValue item : items) {
                    //System.out.println(item.toString());
                    String name = item.asObject().getString("id", "Unknown Item");
                    System.out.println(name);
                   
                    String quantity = item.asObject().getString("url", "id");
                   // JSONArray jsonArray2 = new JSONArray(quantity);
                     System.out.println(quantity);
               
                   /* Platform.runLater(() ->{
                        try {
                            Thread.sleep(10000);
                        } catch (InterruptedException ex) {
                            Logger.getLogger(HV1.class.getName()).log(Level.SEVERE, null, ex);
                        }*/
                    Img.load(quantity);
                            URL url;
    InputStream is = null;
    BufferedReader br;
    String line;
             url = new URL(quantity);
        is = url.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
            //byte[] postData= line.getBytes( StandardCharsets.UTF_8 );
            	//String[] strArray = new String[] {line};
//System.out.println(strArray[0]);
            wb2.load(line);
            String originalUrl = "https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/server/ID";
String newUrl = originalUrl.replace("ID", name);
System.out.println(newUrl);

 URL url1 = new URL(newUrl);
        Map<String,Object> params = new LinkedHashMap<>();
        params.put("content", line);
        params.put("meta", "abc");
       

        StringBuilder postData1 = new StringBuilder();
        for (Map.Entry<String,Object> param : params.entrySet()) {
            if (postData1.length() != 0) postData1.append('&');
            postData1.append(URLEncoder.encode(param.getKey(), "UTF-8"));
            postData1.append('=');
            postData1.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
      
        byte[] postDataBytes = postData1.toString().getBytes("UTF-8");

        HttpURLConnection conn = (HttpURLConnection)url1.openConnection();
        conn.setRequestMethod("POST");
        conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
        conn.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
        conn.setDoOutput(true);
        conn.getOutputStream().write(postDataBytes);

        Reader in1 = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));

        for (int c; (c = in1.read()) >= 0;)
            System.out.print((char)c);

        /*          try{  
       Thread.sleep(400);  
      }catch(InterruptedException e){System.out.println(e);}  */
          }
            }
    } 
    }        
               
            
            VBox vb = new VBox();
            Button btn1 = new Button();
            btn1.setText("Load");
            vb.getChildren().add(btn1);
            Button btn2 = new Button();
            btn2.setText("Download");
            vb.getChildren().add(btn2);
            Button btn3 = new Button();
            btn3.setText("Save");
            vb.getChildren().add(btn3);
            vbox.getChildren().addAll(img, img1);
            vbox2.setPadding(new Insets(3));
            vbox2.setSpacing(3);
            vbox2.getChildren().addAll(browser, vb);

            HBox.setHgrow(vbox2, Priority.ALWAYS);

            hbox.setPadding(new Insets(20));


            hbox.getChildren().addAll(vbox, vbox2);
            root.getChildren().add(hbox);
            Scene scene = new Scene(root, 500, 300); // the stack pane is the root node
            

            primaryStage.setScene(scene);
            primaryStage.show();
         //  btn2.setOnAction(new EventHandler<ActionEvent>() {
            // @Override
           //  public void handle(ActionEvent event) {
                /*  URL url;
    InputStream is = null;
    BufferedReader br;
    String line;
             url = new URL(quantity);
        is = url.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
             }
             }
        });
    }*/
      
    }
}
            
             
    public static void main(String[] args) {
        Application.launch(args);
    }
}

i had solve the problem just the problem is content is a array and meta is also an array which in need to post in content = html body meta = headers in json format how can i do so please help.

[Niti Kapoor]

https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/server/138a28186fff71a6997aab9fa2a1a355 this is url in which i had to post in content
json in url
{"data":null,"errors":["content or meta missing in POST or PUT"],"meta":""}

[Niti Kapoor]

import java.io.*;
import java.net.*;
import java.util.*;

class Test {
    public static void main(String[] args) throws Exception {
         URL url1;
    InputStream is = null;
    BufferedReader br;
    String line;
             url1 = new URL("http://www.bbc.com");
        is = url1.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
        URL url = new URL("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/ping");
        Map<String,Object> params = new LinkedHashMap<>();
        params.put("content", line);
        params.put("email", "[e-mail]");
        params.put("reply_to_thread", 10394);
        params.put("message", "Shark attacks in Botany Bay have gotten out of control. We need more defensive dolphins to protect the schools here, but Mayor Porpoise is too busy stuffing his snout with lobsters. He's so shellfish.");

        StringBuilder postData = new StringBuilder();
        for (Map.Entry<String,Object> param : params.entrySet()) {
            if (postData.length() != 0) postData.append('&');
            postData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
            postData.append('=');
            postData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
        }
        byte[] postDataBytes = postData.toString().getBytes("UTF-8");

        HttpURLConnection conn = (HttpURLConnection)url.openConnection();
        conn.setRequestMethod("POST");
        conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
        conn.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
        conn.setDoOutput(true);
        conn.getOutputStream().write(postDataBytes);

        Reader in = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));

 for (int c; (c = in.read()) >= 0;)
            System.out.print((char)c);
    
    }
}
}

as i had make simple code to understand can you tell me why the complete string is nt post in url at one it is break in to small small parts

[Niti Kapoor]

<!DOCTYPE html> <html class=" b-pw-1280" lang="en" > <head> <!-- Barlesque 3.21.31 --> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <meta name="description" content="Breaking news, sport, TV, radio and a whole lot more. The BBC informs, educates and entertains - wherever you are, whatever your age." /> <meta name="keywords" content="BBC, bbc.co.uk, bbc.com, Search, British Broadcasting Corporation, BBC iPlayer, BBCi" />   <title>BBC - Homepage</title>        <meta name="viewport" content="width=device-width, initial-scale=1.0" />  <meta property="fb:admins" content="100004154058350" />  <script type="text/javascript">window.bbcredirection={geo:true}</script>  
<h2>GET</h2>array (
)<h2>POST</h2>array (
  'content' => ' <!DOCTYPE html> <html class=" b-pw-1280" lang="en" > <head> <!-- Barlesque 3.21.31 --> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <meta name="description" content="Breaking news, sport, TV, radio and a whole lot more. The BBC informs, educates and entertains - wherever you are, whatever your age." /> <meta name="keywords" content="BBC, bbc.co.uk, bbc.com, Search, British Broadcasting Corporation, BBC iPlayer, BBCi" />   <title>BBC - Homepage</title>        <meta name="viewport" content="width=device-width, initial-scale=1.0" />  <meta property="fb:admins" content="100004154058350" />  <script type="text/javascript">window.bbcredirection={geo:true}</script>  ',
  'email' => '[e-mail]',
  'reply_to_thread' => '10394',
  'message' => 'Shark attacks in Botany Bay have gotten out of control. We need more defensive dolphins to protect the schools here, but Mayor Porpoise is too busy stuffing his snout with lobsters. He\'s so shellfish.',
)<h2>FILES</h2>array (
)<h2>HEADERS</h2>array (
  'Content-Type' => 'application/x-www-form-urlencoded',
  'User-Agent' => 'Java/1.8.0_151',
  'Host' => 'inapi.ankiti.com',
  'Accept' => 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2',
  'Connection' => 'keep-alive',
  'Content-Length' => '1166',
)<h2>COOKIE</h2>array (
)<h2>SERVER</h2>array (
  'HTTPS' => 'on',
  'SSL_TLS_SNI' => 'inapi.ankiti.com',
  'CONTENT_TYPE' => 'application/x-www-form-urlencoded',
  'HTTP_USER_AGENT' => 'Java/1.8.0_151',
  'HTTP_HOST' => 'inapi.ankiti.com',
  'HTTP_ACCEPT' => 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2',
  'HTTP_CONNECTION' => 'keep-alive',
  'CONTENT_LENGTH' => '1166',
  'PATH' => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
  'SERVER_SIGNATURE' => '<address>Apache/2.4.18 (Ubuntu) Server at inapi.ankiti.com Port 443</address>
',
  'SERVER_SOFTWARE' => 'Apache/2.4.18 (Ubuntu)',
  'SERVER_NAME' => 'inapi.ankiti.com',
  'SERVER_ADDR' => '172.31.0.104',
  'SERVER_PORT' => '443',
  'REMOTE_ADDR' => '45.115.99.234',
  'DOCUMENT_ROOT' => '/sata1/server/anklib5/Api/Https',
  'REQUEST_SCHEME' => 'https',
  'CONTEXT_PREFIX' => '',
  'CONTEXT_DOCUMENT_ROOT' => '/sata1/server/anklib5/Api/Https',
  'SERVER_ADMIN' => 'webmaster@localhost',
  'SCRIPT_FILENAME' => '/sata1/server/anklib5/Api/Https/api',
  'REMOTE_PORT' => '9610',
  'GATEWAY_INTERFACE' => 'CGI/1.1',
  'SERVER_PROTOCOL' => 'HTTP/1.1',
  'REQUEST_METHOD' => 'POST',
  'QUERY_STRING' => '',
  'REQUEST_URI' => '/api/mms.bulksms101.com/crawler/ping',
  'SCRIPT_NAME' => '/api',
  'PATH_INFO' => '/mms.bulksms101.com/crawler/ping',
  'PATH_TRANSLATED' => '/sata1/server/anklib5/Api/Https/mms.bulksms101.com/crawler/ping',
  'PHP_SELF' => '/api/mms.bulksms101.com/crawler/ping',
  'REQUEST_TIME_FLOAT' => 1513835843.4990001,
  'REQUEST_TIME' => 1513835843,
  'base_anklib' => '/sata1/server/anklib5/Api/Src',
  'base_ank' => '/sata1/server/anklib5/Api',
)<!--[if (gt IE 8) | (IEMobile)]><!-->
<h2>GET</h2>array (
)<h2>POST</h2>array (
  'content' => '<!--[if (gt IE 8) | (IEMobile)]><!-->',
  'email' => '[e-mail]',
  'reply_to_thread' => '10394',
  'message' => 'Shark attacks in Botany Bay have gotten out of control. We need more defensive dolphins to protect the schools here, but Mayor Porpoise is too busy stuffing his snout with lobsters. He\'s so shellfish.',
)<h2>FILES</h2>array (
)<h2>HEADERS</h2>array (
  'Content-Type' => 'application/x-www-form-urlencoded',
  'User-Agent' => 'Java/1.8.0_151',
  'Host' => 'inapi.ankiti.com',
  'Accept' => 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2',
  'Connection' => 'keep-alive',
  'Content-Length' => '341',
)<h2>COOKIE</h2>array (
)<h2>SERVER</h2>array (
  'HTTPS' => 'on',
  'SSL_TLS_SNI' => 'inapi.ankiti.com',
  'CONTENT_TYPE' => 'application/x-www-form-urlencoded',
  'HTTP_USER_AGENT' => 'Java/1.8.0_151',
  'HTTP_HOST' => 'inapi.ankiti.com',
  'HTTP_ACCEPT' => 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2',
  'HTTP_CONNECTION' => 'keep-alive',
  'CONTENT_LENGTH' => '341',
  'PATH' => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
  'SERVER_SIGNATURE' => '<address>Apache/2.4.18 (Ubuntu) Server at inapi.ankiti.com Port 443</address>
',
  'SERVER_SOFTWARE' => 'Apache/2.4.18 (Ubuntu)',
  'SERVER_NAME' => 'inapi.ankiti.com',
  'SERVER_ADDR' => '172.31.0.104',
  'SERVER_PORT' => '443',
  'REMOTE_ADDR' => '45.115.99.234',
  'DOCUMENT_ROOT' => '/sata1/server/anklib5/Api/Https',
  'REQUEST_SCHEME' => 'https',
  'CONTEXT_PREFIX' => '',
  'CONTEXT_DOCUMENT_ROOT' => '/sata1/server/anklib5/Api/Https',
  'SERVER_ADMIN' => 'webmaster@localhost',
  'SCRIPT_FILENAME' => '/sata1/server/anklib5/Api/Https/api',
  'REMOTE_PORT' => '9610',
  'GATEWAY_INTERFACE' => 'CGI/1.1',
  'SERVER_PROTOCOL' => 'HTTP/1.1',
  'REQUEST_METHOD' => 'POST',
  'QUERY_STRING' => '',
  'REQUEST_URI' => '/api/mms.bulksms101.com/crawler/ping',
  'SCRIPT_NAME' => '/api',
  'PATH_INFO' => '/mms.bulksms101.com/crawler/ping',
  'PATH_TRANSLATED' => '/sata1/server/anklib5/Api/Https/mms.bulksms101.com/crawler/ping',
  'PHP_SELF' => '/api/mms.bulksms101.com/crawler/ping',
  'REQUEST_TIME_FLOAT' => 1513835843.592,
  'REQUEST_TIME' => 1513835843,
  'base_anklib' => '/sata1/server/anklib5/Api/Src',
  'base_ank' => '/sata1/server/anklib5/Api',
)<link rel="stylesheet" href="http://static.bbci.co.uk/frameworks/barlesque/3.21.31/orb/4/style/orb.min.css">

[Niti Kapoor]

this is the ouput

[Niti Kapoor]

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package mms;

/**
 *
 * @author niti4
 */
/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */


import com.eclipsesource.json.Json;
import com.eclipsesource.json.JsonValue;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import java.util.LinkedHashMap;
import java.util.Map;
import javafx.scene.control.Button;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.scene.Scene;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.StackPane;
import javafx.scene.layout.VBox;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import org.jsoup.Jsoup;
/**
 *
 * @author biznis
 */
public class HV1 extends Application {

    /**
     *
     * @param primaryStage
     * @throws Exception
     *///
    @Override
    public void start(Stage primaryStage) throws Exception {
        StackPane root = new StackPane();
        HBox hbox = new HBox(30); // create a HBox to hold 2 vboxes        

        // create a vbox with a textarea that grows vertically
        VBox vbox = new VBox(10);
        //Label label1 = new Label("");
        final WebView img = new WebView();
        final WebEngine Img = img.getEngine();
       // Img.load("http://mms.bulksms101.com/api/crawler/downloads");
        final WebView img1 = new WebView();
        final WebEngine Img1 = img.getEngine();
        Img1.load("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/downloads");
        VBox vbox2 = new VBox();
        final WebView browser = new WebView();
        final WebEngine wb2 = browser.getEngine();
        URL oracle = new URL("https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/downloads");
        try (BufferedReader in = new BufferedReader(
                new InputStreamReader(oracle.openStream()))) {
            String inputLine1;
            while ((inputLine1 = in.readLine()) != null) {
                System.out.println(inputLine1);
                com.eclipsesource.json.JsonObject object = Json.parse(inputLine1).asObject();
                com.eclipsesource.json.JsonArray items = Json.parse(inputLine1).asObject().get("data").asArray();
            
                for (JsonValue item : items) {
                    //System.out.println(item.toString());
                    String name = item.asObject().getString("id", "Unknown Item");
                    System.out.println(name);
                   
                    String quantity = item.asObject().getString("url", "id");
                   // JSONArray jsonArray2 = new JSONArray(quantity);
                     System.out.println(quantity);
               
                   /* Platform.runLater(() ->{
                        try {
                            Thread.sleep(10000);
                        } catch (InterruptedException ex) {
                            Logger.getLogger(HV1.class.getName()).log(Level.SEVERE, null, ex);
                        }*/
                    Img.load(quantity);
 String webPage = quantity;

            String html = Jsoup.connect(webPage).get().html();
          //  byte[] postData= line.getBytes( StandardCharsets.UTF_8 );
            	
            wb2.load(html);
            String originalUrl = "https://inapi.ankiti.com/api/mms.bulksms101.com/crawler/server/ID";
            String newUrl = originalUrl.replace("ID", name);
System.out.println(newUrl);
 URL url = new URL(newUrl);
        Map<String,Object> params = new LinkedHashMap<>();
        params.put("content", html);
        params.put("meta", "abc");
       // params.put("reply_to_thread", 10394);
       // params.put("message", "Shark attacks in Botany Bay have gotten out of control. We need more defensive dolphins to protect the schools here, but Mayor Porpoise is too busy stuffing his snout with lobsters. He's so shellfish.");

        StringBuilder postData = new StringBuilder();
        for (Map.Entry<String,Object> param : params.entrySet()) {
            if (postData.length() != 0) postData.append('&');
            postData.append(URLEncoder.encode(param.getKey(), "UTF-8"));
            postData.append('=');
            postData.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
        }
        byte[] postDataBytes = postData.toString().getBytes("UTF-8");

        HttpURLConnection conn = (HttpURLConnection)url.openConnection();
        conn.setRequestMethod("POST");
        conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
        conn.setRequestProperty("Content-Length", String.valueOf(postDataBytes.length));
        conn.setDoOutput(true);
        conn.getOutputStream().write(postDataBytes);

        Reader in1 = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));

 for (int c; (c = in1.read()) >= 0;)
            System.out.print((char)c);
    
    }
}

//writer.close();
//reader.close();           
     //   return responseSB.toString();
    
       // Reader in1 = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));

       // for (int c; (c = in1.read()) >= 0;)
          //  System.out.print((char)c);

        /*          try{  
       Thread.sleep(400);  
      }catch(InterruptedException e){System.out.println(e);}  */
        
        
        
                 
               
            
            VBox vb = new VBox();
            Button btn1 = new Button();
            btn1.setText("Load");
            vb.getChildren().add(btn1);
            Button btn2 = new Button();
            btn2.setText("Download");
            vb.getChildren().add(btn2);
            Button btn3 = new Button();
            btn3.setText("Save");
            vb.getChildren().add(btn3);
            vbox.getChildren().addAll(img, img1);
            vbox2.setPadding(new Insets(3));
            vbox2.setSpacing(3);
            vbox2.getChildren().addAll(browser, vb);

            HBox.setHgrow(vbox2, Priority.ALWAYS);

            hbox.setPadding(new Insets(20));


            hbox.getChildren().addAll(vbox, vbox2);
            root.getChildren().add(hbox);
            Scene scene = new Scene(root, 500, 300); // the stack pane is the root node
            

            primaryStage.setScene(scene);
            primaryStage.show();
         //  btn2.setOnAction(new EventHandler<ActionEvent>() {
            // @Override
           //  public void handle(ActionEvent event) {
                /*  URL url;
    InputStream is = null;
    BufferedReader br;
    String line;
             url = new URL(quantity);
        is = url.openStream();  // throws an IOException
        br = new BufferedReader(new InputStreamReader(is));
       
        while ((line = br.readLine()) != null) {
            System.out.println(line);
             }
             }
        });
    }*/
     
    }
}
            
             
    public static void main(String[] args) {
        Application.launch(args);
    }
}

i had solve this problem but i need outer html

[Knute Snortum]

i had solve this problem but i need outer html  

Are you talking about the outerHTML() method in JavaScript?  What is "outer html"?

[Niti Kapoor]

yes exactly that is outer html() like in js or you can see any website when you inspect it you will see html tags which are not present in view source of that page i hope you can understand

[Niti Kapoor]

document.body.outerhtml() how can we use in jsoup in java

[Knute Snortum]

Niti Kapoor wrote:i hope you can understand

I don't unfortunately.

document.body.outerhtml() how can we use in jsoup in java

It looks like you are using a third party library from https://jsoup.org/ I would suggest that you go there to look for documentation.