• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

SQL Query formation problem.

 
v ray
Ranch Hand
Posts: 223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am supposed to write a query to extract information from a table.
The table has 30 million records. Now, the requirements are:

1. The entire table has to be ordered first based on the id.
2. Once the table is ordered, I have to write a query as below:



Now, I initially formed the query as:

However, this does not work as this only orders the ResultSet from the query. But I need the entire table to be sorted so that if the query breaks in the middle, I need to be able to resume the query from the last id onwards.

Please help!! This is crazy!
 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 34839
369
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by v ray:
But I need the entire table to be sorted so that if the query breaks in the middle, I need to be able to resume the query from the last id onwards.

You can't sort the table. It sounds like you are trying to accomplish saving the database work, which you can do another way.

You can create an index on the id column and order that index (when you create the index, you specify it as a sorted index.) Scanning an index is much faster than scanning the tables which can reduce the overall work.
 
v ray
Ranch Hand
Posts: 223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The table is already indexed, does that help me in any way?
Currently, what I am using is:

select id,xml from xml_hash
where id> startId and rownum < size
order by id;

so if I use 10 threads to execute the query, startId will keep getting incremented which stops at some specified count value...

What do you think of that option?
 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 34839
369
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That will work. Note that some databases do this parallelism for you.
 
v ray
Ranch Hand
Posts: 223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, I am already doing that, but it still needs to be faster, hence the use of threads.

We are currently using:
select id,xml from xml_hash/*+Parallel,4*/
where id> startId and rownum < size
order by id;

We are using a oracle10g database.
Btw, a question. Saying parallel,4 is for the better use of multiple processors right? How do we get the database to use multiple threads, etc to process the above query faster?
The main problem here is, the xml in the above query is a Clob object and is really large!
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic