read and write multiple three character unicode words from a file

[surya preethaaa]

Here the file  read.txt contains multiple unicode words with surrogate pairs ..i m using k=3 in the for loop to get multiple three letter words from the file read.txt..i am having trouble in counting the surrogate pairs..(i.e)the character counts along with diacritic marks...want some ideas to count the surrogate pairs...when running the below code ..i m getting this error...

java.lang.ArrayIndexOutOfBoundsException: 11
at pp.main(pp.java:35)
BUILD SUCCESSFUL (total time: 1 second)

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class pp {

public static void main(String[] args) throws IOException {
    FileReader fr = null;
    BufferedReader br =null;
    FileWriter fw=null;
    BufferedWriter bw=null;

    String [] stringArray;
    int arrayLength ;
  String s="";
   String stringLine="";
    try{
        fr = new FileReader("D:\\OFFICE\\read.txt");
        fw=new FileWriter("D:\\OFFICE\\write.txt");
        br = new BufferedReader(fr);
        bw=new BufferedWriter(fw);
       while((s = br.readLine()) != null)
       {
            stringLine = stringLine + s;
            stringLine = stringLine + " ";
      }
        stringArray = stringLine.split(" ");
  arrayLength=stringLine.codePointCount(0, stringLine.length());
     for (int i = 0; i <arrayLength; i++)  
     {
     for(int k=3;k>stringArray[i].length();i++) //for getting all 3 char unicode lines
              {
          System.out.println(stringArray[i]);
               bw.newLine();
             } 
     int n=stringLine.offsetByCodePoints(0, i);
     arrayLength=stringLine.codePointAt(n);
   }
       fr.close();
        br.close();
        bw.flush();
            bw.close();
    }catch (ArrayIndexOutOfBoundsException e) {
        e.printStackTrace();
    }}}

[Dave Tolls]

    for (int i = 0; i <arrayLength; i++)  
     {
     for(int k=3;k>stringArray[i].length();i++)

This is incrementing i in the outer loop.
However you are also incrementing i in the inner loop.

Now, arrayLength seems to have been set based on the number of codePoints?
I would stick some debugging in there as the value is likely incorrect.
Also stick some debugging around the value of i in these two loops.

[surya preethaaa]

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class to {

public static void main(String[] args) throws IOException {
    FileReader fr = null;
    BufferedReader br =null;
    FileWriter fw=null;
    BufferedWriter bw=null;

    String [] stringArray;
    int arrayLength ;
  String s="";
   String stringLine="";
    try{
        fr = new FileReader("D:\\surya\\read.txt");
        fw=new FileWriter("D:\\surya\\write.txt");
        br = new BufferedReader(fr);
        bw=new BufferedWriter(fw);
       while((s = br.readLine()) != null)
       {
            stringLine = stringLine + s;
            stringLine = stringLine + " ";
      }
        stringArray = stringLine.split(" ");
       arrayLength=stringLine.length()-1;
  for (int i = 0; i <arrayLength;i++)  
    if (stringArray[i].length() == 3)
    {
         System.out.println(stringArray[i]);

         bw.newLine();
    }
   fr.close();
        br.close();
        bw.flush();
            bw.close();
     }catch (ArrayIndexOutOfBoundsException e) {
        e.printStackTrace();
    }}}

i tried this coding....that count the character correctly in english language ....it creats problem wen using file with unicode words with surrogate pairs...

[Campbell Ritchie]

Don't call close() on your readers; use try with recources instead. Don't call flush() and close() on your writers; use try with resources instead.
Doesn't String have a method for counting code points? Surely that will solve your problem with two chars to one Unicode character.

[surya preethaaa]

ok....then i try this coding  using codePointCount()

[praveen kumaar]

Hi surya,can you please explain a bit more about your expectation from the above program.

[surya preethaaa]

in english ;

if the file contains

HELLO
HOW
IS
YOUR
WORK

if i use - if (stringArray[i].length() == 3)
it writes the word

HOW to the write.txt file

while using tamil language files :

eg) if a file containg words

பலன்கள்
சூரியன்
தொழில்
பலன்

if i use - if (stringArray[i].length() == 3)

it should write the word பலன் to write.txt file..

but it will throw exception errors

main problem is
பலன் -counting as four character along with dot..
i want to count it as three character

[Campbell Ritchie]

You need to remove punctuation from the text if you want to do that.

[surya preethaaa]

பலன் is a single word in tamil language...
if dot is removed...it became meaningless

[Stephan van Hulst]

You're moving in VERY treacherous area.

There is no agreed upon meaning of the word "character". It was probably a mistake by the Java designers to call char a char, and to call Character a Character. They don't represent characters, they represent UTF-16 code units.

What is a character? Do you mean a code unit? A Unicode code point? A grapheme cluster?

ன் is one grapheme cluster. It consists of two unicode code points. There are NO surrogate pairs involved, so each of these code points is represented by ONE UTF-16 code unit.

Sadly, I don't know how to easily count grapheme clusters in Strings using Java, or if there are good libraries to do this.

[Stephan van Hulst]

You could try to use javax.swing.text.PlainDocument for this. Put your text in this document and see how many characters it reports.

I tested this, sadly it doesn't work.

[Norm Radder]

Also posted here: http://www.dreamincode.net/forums/topic/398165-read-and-write-multiple-three-character-unicode-words-from-a-file/
Please read: https://coderanch.com/wiki/660346/Forthright-Cross-Posting-Sites

[Campbell Ritchie]

Stephan van Hulst wrote:. . . There is no agreed upon meaning of the word "character". It was probably a mistake by the Java designers to call char a char, and to call Character a Character. They don't represent characters, they represent UTF-16 code units. . . .

They copied the name from C/C++; a char in C is different, only occupying 8 bits. When Java® was introduced, Unicode only went up to 16 bits, so things like code points are later additions which are much more awkward than they would have been if >16‑bit Unicode had been around then.

[Stephan van Hulst]

Well, Unicode itself isn't much less awkward either. Character combining is done regardless of which UTF encoding you use.

Any method of String that accepts or returns an index or a length should be replaced by a bunch of methods that do the same, but make clear whether the method operates on code points or grapheme clusters. For instance, there should have been a codePointAt() method that returns a CodePoint, and a clusterAt() method that returns a GraphemeCluster.

It might make the API look more difficult and awkward to use, but that's a good thing, because at least it reflects the fact that String handling IS difficult and awkward.

[surya preethaaa]

sorry i jus mentioned the GraphemeCluster as a surrogate pair....i am trying to count unicode without GraphemeCluster

[Stephan van Hulst]

In that case, what is wrong with how it currently works?

பலன் consists of 4 code points without surrogate pairs. That means that length() will return 4, and that the string will not appear in your output file.

Why do you want it to appear there, because ன் appears as one character? That's because it's one grapheme cluster, consisting of two code points.

[surya preethaaa]

actually i m trying to read a words from a file containing 3 letters and write those words into another file....in tamil language, பலன் is a single word ....if i remove dot...it became meaningless...so i want to count பலன் as 3 and should write into another file..(i.e i want to count the words without considering its grapheme clusters)

[surya preethaaa]

some single words with grapheme clusters

பலன்கள்-ப,ல,ன்,க,ள்

சூரியன்-சூ,ரி,ய,ன்

[surya preethaaa]

if fileA contains the words

பல
ராசி

eg)if i want 2 letter words from the fileA..
this code jus get பல ...
its not getting ராசி..which is also a 2 letter word (i.e)ரா,சி

[surya preethaaa]

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.FileReader;
import java.io.FileWriter;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class pp {

public static void main(String[] args) {
    FileReader fr = null;
    BufferedReader br =null;
    FileWriter fw=null;
    BufferedWriter bw=null;

    String [] stringArray;
    int arrayLength ;
  String s="";
   String stringLine="";
    try{
        fr = new FileReader("D:\\OFFICE\\jj.txt");
        fw=new FileWriter("D:\\OFFICE\\o.txt");
        br = new BufferedReader(fr);
        bw=new BufferedWriter(fw);
      while((s = br.readLine()) != null){
            stringLine = stringLine + s;
            stringLine = stringLine + " ";
      }
        stringArray = stringLine.split(" ");
        arrayLength = stringArray.length;
       
     for (int i = 0; i < arrayLength-1; i++) {
      
              for(int k=2;k==stringArray[i].length();i++)
              {
              bw.write(stringArray[i]);
               bw.newLine();
              
              }   }                     bw.close();
    }catch (Exception e) {
        e.printStackTrace();
    }}}

[Stephan van Hulst]

Like I said, then you shouldn't use String.length(), because that counts UTF-16 code units; NOT Unicode code points, and also NOT grapheme clusters. You're interested in the amount of grapheme clusters in the String.

There is no easy way to get the number of grapheme clusters in a String. You'll have to look for a library that gets these, or you'll have to write an algorithm to do this.