Java – Reading From a URL using URLConnection Class

URLConnection is an abstract class whose subclasses form the link between the user application and any resource on the web. We can use it to read/write from/to any resource referenced by a URL object. There are mainly two subclasses that extend the URLConnection class

• HttpURLConnection: If we are connecting to any URL which uses “http” as its protocol, then HttpURLConnection class is used.
• JarURLConnection: If however, we are trying to establish a connection to a jar file on the web, then JarURLConnection is used.

Once the connection is established and we have a URLConnection object, we can use it to read or write or get further information about when was the page/file last modified, content length, etc. 

But barely getting the state information is not the true motive of a real-world application. To retrieve the information, process it, and send the results back to the server, or just display the required information retrieved from the server is what we are aiming at. 

Illustration:

Small scale application which asks for a movie name from the user and in turn returns the “IMDb” rating of the movie or returns all the links related to that movie. All of this can be achieved using the URLConnection class.

Methods of URLConnection Class

Method	Action Performed
getContent()	Retrieves the content of the URLConnection
getContentEncoding()	Returns the value of the content-encoding header field.
getContentLength()	Returns the length of the content header field
getDate()	Returns the value of date in the header field
getHeaderFields()	Returns the map containing the values of various header fields in the HTTP header
getHeaderField(int i)	Returns the value of the ith index of the header
getHeaderField(String field)	Returns the value of the field named “field” in the header
getInputStream()	Returns the input stream to this open connection been inside of OutputStream class
getOutputStream()	Returns the output stream to this connection of OutputStream class
openConnection()	Opens the connection to the specified URL.
setAllowUserInteraction()	Setting this true means a user can interact with the page. The default value is true.
setDefaultUseCaches()	Sets the default value of useCache field as the given value.
setDoInput()	Sets if the user is allowed to take input or not
setDoOutput()	Sets if the user is allowed to write on the page. The default value is false since most of the URLs don’t allow to write

Now after having an understanding of the methods Steps involved in the above process 

1. URL Creation: Create a URL object using any of the constructors given
2. Create Object: Invoke the openConnection() call to create the object of URLConnection.
3. Display the Content: Either use the above-created object to display the information about the resource or to read/write contents of the file to the console using bufferedReader and InputStream of the open connection using getInputStream() method.
4. Close Stream: Close the InputStream when done.

Implementation: Let us look at a sample program, which uses the above methods to display the header fields and also print the source code of the entire page onto the console window.

Java

// Java Program to Illustrate Reading and Writing
// in URLConnection Class

// Importing required classes
import java.io.*;
import java.net.*;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

// Class
// URLConnectionclass
class GFG {
    // main driver method
    public static void main(String[] args)
    {

        // Try block to check for exceptions
        try {

            // Fetching URL by creating
            URL url = new URL(
                "https://www.geeksforgeeks.org/java");

            // Opening the connection to the above URL
            URLConnection urlcon = url.openConnection();

            // Executing the below line would print the
            // value of
            // Allow User Interaction field.
            // System.out.println(urlcon.getAllowUserInteraction());

            // Executing the below line would print
            // the value of Content Type field.
            // System.out.println(urlcon.getContentType());

            // Executing the below line would print the
            // value
            // of URL of the given connection.
            // System.out.println(urlcon.getURL());

            // Executing the below line would
            // print the value of Do Input field.
            // System.out.println(urlcon.getDoInput());

            // Executing the below line would
            // print the value of Do Output field.
            // System.out.println(urlcon.getDoOutput());

            // Executing the below line would
            // print the value of Last Modified field.
            // System.out.println(new
            // Date(urlcon.getLastModified()));

            // Executing the below line would
            // print the value of Content Encoding field.
            // System.out.println(urlcon.getContentEncoding());

            // To get a map of all the fields of http header
            Map<String, List<String> > header
                = urlcon.getHeaderFields();

            // Printing all the fields along with their
            // value
            for (Map.Entry<String, List<String> > mp :
                 header.entrySet()) {
                System.out.print(mp.getKey() + " : ");
                System.out.println(
                    mp.getValue().toString());
            }

            System.out.println();
            System.out.println(
                "Complete source code of the URL is-");
            System.out.println(
                "---------------------------------");

            // Getting the inputstream of the open
            // connection
            BufferedReader br
                = new BufferedReader(new InputStreamReader(
                    urlcon.getInputStream()));
            String i;

            // Printing the source code line by line
            while ((i = br.readLine()) != null) {

                System.out.println(i);
            }
        }

        // Catch block to handle exceptions
        catch (Exception e) {

            // Displaying exceptions
            System.out.println(e);
        }
    }
}

Output:

Keep-Alive   :   [timeout=5, max=100]
null   :   [HTTP/1.1 200 OK]
Server   :   [Apache/2.4.18 (Ubuntu)]
Connection   :   [Keep-Alive]
Last-Modified   :   [Wed, 16 Nov 2016 06:49:55 GMT]
Date   :   [Wed, 16 Nov 2016 10:58:34 GMT]
Accept-Ranges   :   [bytes]
Cache-Control   :   [max-age=3]
ETag   :   ["10866-541657b07e4d7"]
Vary   :   [Accept-Encoding]
Expires   :   [Wed, 16 Nov 2016 10:58:37 GMT]
Content-Length   :   [67686]
Content-Type   :   

Complete source code of the URL is-
--------------------------------------------------

...source code of the page...

This article is contributed by Rishabh Mahrsee. If you like GeeksforGeeks and would like to contribute, you can also write an article using write.geeksforgeeks.org or mail your article to review-team@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks. Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.