Abhinav Srivastava

Ranch Hand
+ Follow
since Nov 19, 2002
Merit badge: grant badges
Cows and Likes
Cows
Total received
In last 30 days
0
Forums and Threads

Recent posts by Abhinav Srivastava

UUID's are not very user friendly. Consider something like a SSN or a Bank Acct#. (I guess they are not truly random but just for example's sake)...
Been scratching my head over this. I am trying to find a way to generate a

1. Number having a fixed length (no of digits)
2. Unique but not easily guessable (that rules out things like DB generated seq numbers).

It seems like a some sort of bijection function would foot the bill.

Thoughts?
Is there a way on a windows server (using in-built commands) to watch the network connectivity (to a specific IP) speed or outages over a given period of time? We are trying to debug some intermittent performance issues and do not have any commercial tools available yet.
9 years ago

nawab wajid wrote:I have fair understanding of the hadooop and MapReduce concepts .but still I need to understand these concepts on a higher technical level. I know that NameNode stores the metadata information about the data and DataNode stores the actual data.BUT I am not understanding what makes the node as NameNode and Data Node? i mean what special things do we need to make it a NameNode and DataNode?



There are configuration files that specify the slave/data nodes.

nawab wajid wrote:
2)What is the difference between HDFS and local file system in hadoop ? I have understood that the mapper output is stored in local file system and the reducer output is stored on the HDFS. Also if there is no reducer then the mapper output is stored directly on the HDFS .What I am not understanding is where is HDFS and where is local file system located? How do I differentiate between the two as far as hadoop is concerned?



HDFS and Local Filesystems are non-overlapping. However, the File I/O is generic so you could use that to write to local FS as well as HDFS (see scheme in fs.default.name). The HDFS would not even be visible to the native 'ls' command.
9 years ago
Do we need both -libjars and setJarByClass() to set the proper classpath for tasks or either would do?

If the driver and MR code are all in the same jar and assuming there is no third party dependency (other than hadoop framework) do we still need to set jars in class path using libjars or setJarByClass()?


9 years ago
The INSERT/UPDATE/SELECT sql are within the application and to change them to use ENCRYPT/DECRYPT functions whenever the columns in question are mentioned will have a huge impact. It may not be possible cosidering its a 3rd party application.
10 years ago
We have an exisitng 3rd party J2EE application on a DB2 (AIX) database and the policy requires that a few columns such as SSN, DoB be encrypted while 'at rest'. I have explored a few options such as the ENCRYPT/DECRYPT functions and TRIGGERS/VIEWS however it seems that they all require some changes to the application. I have come across Field Procedures (FIELDPROC), not sure if it is available outside IBM i systems. Any ideas?
10 years ago

Or do you mean that you don't want to buy that much idle hardware, and "borrow" it only when you need it?

Exactly.

Though Amazon-like cloud would be the best, under the constraints we have, its a decision between scale up and/or scale out. As we are using LPARs, it does give a little room for scaling up.
10 years ago
We are working on determining an in house hardware configuration using WAS (8 Core AIX, 50 GB mem) /DB2 (4 Core AIX, 32 GB mem) servers running a JEE Application.
There would be corresponding failover servers.
What we foresee is that after the system has gone live, there would be a need of server capacity to be scaled up to accommodate bulk processing of data, but this increased capacity (in terms of Cores/Memory) will only be required for
a very short duration e.g. a week or two.. twice or thrice a year, every year.

My question is how we can scale-up and scale-down the capacity on demand with minimal impact to a running system.
One option on the table is to introduce a new AIX+WAS working on the same Database and use that exclusively for the bulk processing.
Any other options?
10 years ago
So I was reading about the MBPr and its unmatched resolution. One question comes to mind, as more pixels are crammed into say a 14" display, the size of any object would reduce.
A 1080p frame from a video would have the resolution of 1920 × 1080 as opposed to 2560x1600 of MBP.
To make full use of the real estate on the display, one would have to upscale. Would the quality (theoritically) suffer as a result of upscaling? Which display would fare better - 1920x1080 or 2560x1600?
11 years ago
I have a multithreaded application where each thread logs messages using a code similar to this:



What I have observed is that this results in creation of multiple files named like
application.log.N as well as application.lock.N irrespective of FileHandler.count value in logger properties.



What could be going wrong here?
12 years ago
Worked with explicitly set PATH


12 years ago
Trying to execute Java from a shell script, any clues?


12 years ago
My web application is using a PKCS12 keystore on tomcat. The client program requires a PEM format certificate. So if I convert this keystore to PEM and give the certificate to client while the server keeps using PKCS12 would that work? Or do we need the same format on both ends?
12 years ago
JNDI is typically used for looking up resources such as EJBs and DataSources. JNDI API is only an abstraction over the underlying directory (e.g. LDAP).
In a typical ServiceLocator implementation, you do cache looked-up objects in a HashMap. But you still use the JNDI API to access objects first time.

If you application requires to lookup same objects again and again, such as a configuration, it can very well be kept in a global (such as Application Context) HashMap. This HashMap cannot be (directly) looked up from other JVMs, however an object bound to JNDI tree can be looked up from other JVMs also.