It can be done, but it's going to require a bit of extra work. First of all, you need to know where and how the file association is stored. Then you need to know how to create / modify associations. And you may also want to have an undo function that allows you to restore the previous file association, in case someone wants to uninstall your application.
when installing ask user if you want to associate .txt with your program. to start with you can make a new extn like .t1 and use that
most installers have the API to call to associate an extension with a program - some of the freeware ones too.
Your program will have to a .bat to start with
which will call java command line and pass the text file name to it
java.exe com.your_package.your_main_class %*
for this to work java should be installed and be in windows PATH var
else need to add the java home path ...
c:\java_home_path\bin\java.exe com.your_package.your_main_class %*
one level deeper would be to make a custom java launcher exe but that will be in c++ - check where java is installed and then launch it with correct params