Suvojyoty Saha

Sorry for the late reply Jesus. I guess we have returned to the beginning of our discussion.

Thanks

Thanks Jesus

I understand what you are trying to say. Informatica is an excellent ETL tool. In similar perspective we can also use SQL Developer or TOAD to read the files and insert onto the database. The data format is extremely important for both.

But my question remains the same instead of using these tools if i have to operate with only Java what is the best way to do so.

hehe. Thanks Jesus for the support.

but seriously is there any better way of increasing the performance by loading chunks of data instead of loading each line and inserting onto the database?

I understand Jesus. But data format can come into picture only after reading a line through BufferedReader.

But how to speed up the reading process itself?

i know Mathew. I guess the interviewer wanted to retrieve all the info as quickly as possible.

One way i could think of as to increase the speed slightly was by introducing threads.

1. one thread to read each line and insert into a collection
2. the second one using the collection to insert the values onto the database.

Apart form this i am not sure as to how to increase the speed.

I have always used

File myFile = new File(pathName); FileReader fileReader = new FileReader(myFile); BufferedReader reader = new BufferedReader(fileReader);

the above code for reading the file and then use reader.readLine() for reading each line of the file.

But many interviewers have asked the question as to how to tackle a scenario if we had to read millions of records.

what should be the answer to the above question? it will be very kind if anybody can provide me some links where i can increase my knowledge base.

thanks,
Suvojyoty

Tim Holloway - I too am not sure as to why the interviewer kept asking me such questions But thanks for the answer.


Tim Moores - Thanks a lot for the answer

Thanks Tim. But how should i answer such questions?

This was the question which was asked to me in an interview.

I replied saying there are many functionality incorporated in struts which helps us utilize them such as

1. if there are any mapping changes that needs to be performed we can change those in struts-config.xml and deploy it the change would be reflected unlike in web.xml where we have to restart the application.

The interviewer replied saying what if we have a tool in the future which would enable hot deployment, how would struts be different from servlets then.

2. I said, the validations can be performed collectively without re-writing of code in several servlets.

The interviewer replied saying that it does not bother him for the same validation code being used in several servlets.

3. I said, with the help of ActionForm we can auto populate the data and use it in our action class and process that data.

The interviewer replied saying he would use some other third-party tool to accomplish the same.

4. I replied saying all these functionality are combined in the Struts framework.

The interviewer replied asking what functionality in the struts-config xml file cannot be duplicated? - and i was not able to say anything more.

Could anyone be able to help me as to what answer the interviewer needed.

Sorry Wendy I was not sure as to what I was doing. Was not aware that :thin@ needed to be mentioned. Was trying to use the OCIConnection class instead of trying to the driver class from the jar ojdbc14.

Thanks Swastik.

On changing these 2 lines of code i was able to connect.

String urldbName = "jdbc:oracle:thin:@localhost:1521/XE"; String driver = "oracle.jdbc.driver.OracleDriver";

Hi I am trying to test the connection for oracle XE (10g) in Eclipse but i am getting the following Exception

java.sql.SQLException: [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified

I am using the following 10g resease 2 jar ojdbc14.jar and my code is Listed below

package com.test.conn; import java.sql.Connection; import java.sql.DriverManager; import java.sql.SQLException; public class ConnOracle { public static void main(String[] args) { Connection loginConn =null; System.out.println("trying connection\n\n"); try { String urldbName = "jdbc:odbc://localhost:1521/XE"; //XE is the SID as per mentioned in tnsnames.ora String driver = "oracle.jdbc.oci.OracleOCIConnection"; String userName = "SYSTEM"; String password = "admin"; Class.forName(driver); loginConn = DriverManager.getConnection(urldbName,userName,password); System.out.println(loginConn); } catch (SQLException e) { // TODO Auto-generated catch block e.printStackTrace(); System.out.println("Exception Encountered"); } catch (ClassNotFoundException e) { // TODO Auto-generated catch block e.printStackTrace(); System.out.println("Exception Encountered"); } finally { try { loginConn.close(); System.out.println("Out of connection"); } catch (SQLException e) { // TODO Auto-generated catch block e.printStackTrace(); System.out.println("Exception Encountered"); } } } }

I created a USER DSN for Oracle XE and there the connection tests fine. The following driver if replaced with MYSQL Driver also tests fine but not able to get a clue as to what i am doing wrong here.

Thanks a lot Matthew. I think that is the problem. Thanks a lot.

You are right Neal. I can just print the output. Thanks a lot.

I am able to see all the data in my arraylist but when I am trying to write it onto a .csv file why is it not writing all the data?

The code is listed below:
FileWriter allRecords = new FileWriter("D:/testfiles/tempfiles.csv"); Iterator eachRecord = treeRecords.iterator(); while(eachRecord.hasNext()) { ArrayList tempList = (ArrayList) eachRecord.next(); int length = tempList.size(); Iterator tempEachRecord = tempList.iterator(); int count1 =0; while(tempEachRecord.hasNext()) { count1++; String finalList = (String) tempEachRecord.next(); allRecords.write(finalList); allRecords.write(","); if (count1 == length) { allRecords.write("\n"); } } }

Hi I have a csv file which has 1977*15 cells. One of the cells is Authors which has more than 1 author hence the data is in the format ("a,b,c") where a,b,c are different authors. My task is to get the value in all the cells, sort it (based on Authors and Document Title) and then insert it into the database.

But even before sorting the values and inserting the data onto the database i am trying to check if I have the complete data in my arraylist. hence transferring the whole data onto a self created csv file.

Its here on this newly created csv file that i am not able to view the complete data and the data retrieved keeps on changing as I try to implement different logic. Is there a bottleneck on the Arraylist while storing the data? I'm using eclipse and there is no Out of Memory error which is being shown.

Thanks Bear I'm planing to use sessions soon as a login page needs to invoke sessions. Thanks a lot once again.