Yes, most DBs will allow youto 'slice and dice' the result set so that you only need to fetch and return the records that you really want. PostgeSQL, for example, uses the LIMIT and OFFSET keywords to specify a result set 'slice'.
I am not familiar with any freeware tags that do this as I have created my own.
It's not really all that hard to do yourself though. You submit a request that specifies the 'page number' and number of records per page, and the run the query using limits computed from those values. I generally take the DB results and convert it to an Object array to send to the page for display using nested <c:forEach> tags to construct the HTML table.