Nilesh Chhapru

Tim,

Thanks for confirmation

Thanks to ranch staff and the publisher, and congrats to other winners.

Just a quick question, do i need to send an email from email service available on java ranch or i can use any of the email service to send my details.

Also, email service available on java ranch isn't able to find the user id "bookpromotion", please assist.

Regards,
Nilesh Chhapru

Sean,

Thanks for replying.

Yes, what you said is correct and is as per the documentation of storm, but when you have less buffer size and fetch size.. and the message you are trying to process is huge in size.. the you max spout pending wont work as per you have configured, since the buffer size is small and can pass only one or max 2 messages at one time.

I just want to know if we can draw a mathematical relation between both the configurations.

Regards,
Nilesh Chhapru.

Storm is a fault tolerant framework, where all the messages are ack once they complete the flow in topology, which isn't the case in Hadoop.

Moreover the nimbus and supervisor are not tightly coupled unlike hadoop task tracker and job tracker, i.e the supervisor will still keep on processing the data if the nimbus go down.. and all the workers will complete the task allocated, and once nimbus is back it will work as if nothing happened.

Hi All,

We are using storm Kafka integration where a Spout reads from a Kafka topic.

Following is the version of storm, Kafka and zookeeper we are using.
Strom : apache-storm-0.9.2-incubating
Kafka : kafka_2.8.0-0.8.1.1
Zookeeper : zookeeper-3.4.6

I am facing following issues at spout.
1)The messages gets failed even if the average time taken is less than max.topology.timeout value, also we aren’t getting any exceptions at any of the bolt.
2)A topology is finally emitting to the Kafka producer i.e. some other topic, but the messages are getting duplicated due to replay issues.
3)The consumer group is isn’t working properly for storm Kafka integration.
a.When we give same group id to the Kafka consumer of different topology but still both are reading same messages.
b.If we have 2 different consumer with different consumer group id in different topology it works fine if both topologies are deployed at the same time, but doesn’t if we deploy one of them after some of the message are already loaded in the topic and read by the first topology.

Kindly help me with above points as it is hampering the overall scope of the project and also time lines.

Regards,
Nilesh Chhapru.

Hi All,

I am facing Netty reconnect on my worker log, can anyone please let me know if this cause any issue.

Also, at times when the number of connections exceeds the limit of retries it gives some nio, exceptions.

Please let me know how to avoid these logs and issue on worker nodes.

Regards,
Nilesh Chhapru.

Hi All,

When I Increase the KafkaConfig.bufferSizeBytes & KafkaConfig.fetchSizeBytes values to (100 * 1024 * 1024), I get the desired throughput, but the spout after sometime or whenever I restart the topology goes OUT OF MEMORY, but when I reduce the value of Bytes to (5 * 1024 * 1024) then it impacts the throughput largely.

I also wanted to know the relation between Max Spout Pending VS (KafkaConfig.bufferSizeBytes & KafkaConfig.fetchSizeBytes), does batch size is decided with Max Spout Pending Config or with KafkaConfig.bufferSizeBytes.

Regards,
Nilesh Chhapru.

its still a doubt to me....

i am also not able to know that where is the problem in the architecture...
can you guide me on to how to know the exact issues in step by step...


Regards,
Nilesh Chhapru.

Thanks for the reply....

I have used jmap command on the glassfish process id and it gives high memory for all persistence classes.
Also, if there is issue with the code or Application then that would behave the same after i deploy and not after 4-5 hrs with 30-40 users.

Regards,
Nilesh Chhapru.

Hi all,

I am facing memory problem when after the J2EE application is running for more than 3-4 hrs with 20-40 users.
I have a 3-Tier J2EE application distributed as under

1) Data Access Layer (JPA 2.0 with Eclipse Link - EJB 3.1).
2) Business Logic Layer (EJB 3.1)
3) Presentation Layer (JSF 2.0).
Server Used is Glassfish 3.0.1 on windows 2003 server.

Kindly suggest as the memory used by Glassfish Process is insane.

Regards,
Nilesh Chhapru