Granny's Programming Pearls
"inside of every large program is a small program struggling to get out"
JavaRanch.com/granny.jsp
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Paul Clapham
  • Knute Snortum
  • Rob Spoor
Saloon Keepers:
  • Tim Moores
  • Ron McLeod
  • Piet Souris
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Tim Holloway
  • Frits Walraven
  • Ganesh Patekar

use of clusters?

 
Ranch Hand
Posts: 188
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
this was maybe asked in the past - so feel free to direct me to topics

I just came up a few videos on YT about clusters of one-board-systems like Raspberry Pi and others and have to ask: nice, but ... WHY?
I'm aware of what supercomputers are used for like weather forecast, nuclear physics or even scanning space for live - but what could I use a cluster of such SoCs for? Even combined thier computational power isn't that great - and if I think about the electricity bill - I guess using my FX-8350 + R9 290x system can get done work for less money (based on each Pi use up to 5V x 2A = 10W) when considered (FL)OPS per Watt.

Just enlighten me about your "1st world problems" a nerd in its end-20s without wife or kids would benefit from spent the cost to build and run a cluster instead of using his half-decade old gaming-rig ... I just don't get it.
 
Saloon Keeper
Posts: 10428
223
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Fun?
 
Marshal
Posts: 65108
247
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Portability?
 
Rancher
Posts: 93
6
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's purely educational in my opinion. If you want to learn about distributed computing/big data - a cluster of Raspberry Pi is one way of doing that (and playing with toys is fun!). The poor specs of a Pi (especially RAM) can only help with that.
 
Campbell Ritchie
Marshal
Posts: 65108
247
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You mean, if you can program to a π you can program to anything? Hahahahahahahahaha!
 
Bartender
Posts: 20938
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Funny you should mention the Raspberry Pi.

Athough best known for the credit-card and gumstick form factors, there's a third type of Pi: the compute node, which is designed specifically to be clustered and plugged into a backplane.

In fact, the official Raspberry Pi website runs on a battery of compute nodes. So clustering the Pi isn't just a fun exercise, it's a practical reality. And, aside from bragging rights, if I recall correctly, the Pi website cluster is physically smaller/less power than, say, a more traditional blade server system.
 
Matt Wong
Ranch Hand
Posts: 188
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I didn't meant it platform specific just to Raspberry Pi but to basicly any cluster of any type - maybe even mixed ones. I just used RasPi as kind of placeholder as it's most popular on YT.
Aside from the Pi site itself runs on such a cluster I'm more specific into personal use.
Back in the early days of crypto currency man used those SoCs just to privde some way to hook up usb-hubs for those usb-erupters - but I guess unless you already have a solar system (or any other/additional form of harvest energy from natural resources like constant flowing river or a small wind turbine) up and made enough before the crash to break even the upfront invest it's not worth anymore today as you still would pay more for electricity than you would get from your mined coins.
I can see it maybe used as "learning excersise" - but then what? What is it you as a private would then do with it you learned from it? This some sort of inversed chicken-egg problem: Set up to learn how to use it but then after learning not know what to use it for?!
 
Tim Holloway
Bartender
Posts: 20938
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Actually, for the Pi 3, I think it was closer to 5W/node. Not sure about the Pi 4.

I have no doubt that people are finding all sorts of amusing things to do with clustered SBCs.  I think that the on-board video processor alone is worth something and a lot of people these days are into Edge Computing. You can already do some interesting things on the edge with just 1 PI, but take 5 of them and you're still talking less wattage than a tower PC. And less space.
 
Matt Wong
Ranch Hand
Posts: 188
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I can understand the "(FL)OPS per Watt per volume" wich raises the overall efficiency factor - but for the workload I can think of (live video transcoding) they're still not powerful enough
another scenario I can think of would be a distributed raid - wich depend on what and how many drive interfaces each SoC provides (some are up to 4x s-ata2) - but a well designed (like with internal redundancy) nas may still perform way better although more expensive

as you see: yes, although transcode/render or storage clusters possible I doubt they have more to offer than just a proof-of-concept as they just lack raw compute power
 
Tim Holloway
Bartender
Posts: 20938
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Don't assume too much on video transcoding. The mplayer/ffmpeg software available on a Pi (and other Linux systems) is quite capable. Also, the power of the CPU isn't the determining factor for video on the Pi if you use the very-capable on-board video chip. A single Pi - even a Pi Zero - can do real-time video image recognition. In fact, that's a favorite use for the Pi computers and it's not too hard to set up.

A NAS system is a different matter. A NAS does not need a lot of CPU power, since its primary function is to route data, not chew on it. So clustering Pi's would have little benefit. While another popular use for the Pi is as a NAS controller, I think most people manage with a single Pi. It's not an ideal performance server, since there is no built-in hard drive interface (you have to use USB) and the wired network interface for the Pi 3 and below never was all that fast. That's a selling point on the Pi 4, in fact: true gigabit Ethernet. Still, you don't need ultra-high performance to stream broadcast-quality video over a LAN or even fast SATA, so people use Kodi or one of its relatives on the Pi for entertainment centers all the time.

Actually, I've set up a Pi 3 with Kodi and a USB video stick as an emergency TV. Unlike the pocket NTSC TV sets that were quite cheap, getting a small digital TV that can easily run on batteries for, say, the 10 hours or so it takes a hurricane to pass over is difficult and/or expensive. So I set up a Pi TV. And since the serious hurricane season is just now firing up, it may see some use shortly.

So in sum, a SBC can do quite a bit even before clustering. I use the Pi for my examples, as it's a popular and easy-to-use favorite, although I have other devices as well. SBC clusters are good because they are cheap to expand and made up of off-the-shelf parts. They're not the solution to every problem, but they're an option worth considering. Also, having lots of nodes is sometimes more important than having high-performance nodes, and in that case, again, the small size and low power consumption of SBCs is a plus.
 
Tim Holloway
Bartender
Posts: 20938
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Here's an example of how serious some people are about SBC clustering:

https://www.ebay.com/itm/8-Slot-Cluster-Cloud-For-Raspberry-Pi-4B-and-other-single-boards/123315692330

An off-the-shelf cluster frame.

And even an entire business based on SBC clustering: https://www.picocluster.com/
 
Matt Wong
Ranch Hand
Posts: 188
5
MS IE Notepad Suse
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm aware that such products exists to serve the pro-sumer market - but to keep to the mentioned example about live stream encoding:
I'm not sure about that, but I'm sure that none of the existing affordable SoC/SoB in the range up to 100 bucks can handle a full hd 1920x1080 stream with 60 fps (we can neglect the audio as this doesn't need much) - even my decent gpu struggles from time to time (depend on how much change from one frame to another is going on) with this load. If one would cluster a bunch of said systems, they may have the raw power - but there's added overhead for split the incoming signal to the next free node and to re-combine the transcoded stream in order - wich each has to be done on its own. So a small cluster I could come up with would consist of at least 4 units:
1 for receiving the stream (no matter if hdmi, dvi, dp, vga, s-video, component, scart, etc ...) and splitting it to the work nodes
2 for splitting up the work load
1 for recombine the transcoded data from the work nodes and send to its destination
I could only see increase the middle stuff by so far until one hit the limit of one of the two outside nodes handling the splitting and combine. But, the effort I would had to put into such a project - aside from learning "cool, can be done this way" - I would use normal consumer pc components just for thier raw power. That's why I can't see these small not-so-much-power-SoC/SoB would scale for such use cases. Sure, if one replace the tiny nodes by big server multi-cpu/-gpu server stuff - I can see the scaling as the workload for splitting would be way less than the outer nodes are able to handle - so it could be used not just for one but for many streams.

I don't want to say these on-chip-boards are useless - in fact, I myself use a pcengines APU2 for my small back up mail server - and this one job is done pretty well - but it's also very high workload (somewhere in the upper 50% than the lower 50%) for this small 4 core AMD with 1 G ram for each core (I don't know if the ram is shared as one 4G between all 4 cores or if they dedicated - but it's measureable slower when data transfered from one core to another than when both are processed by the same core). I just can'T see why someone would bother and cluster them up for thier private home use aside from "learning practice".
 
Tim Holloway
Bartender
Posts: 20938
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I do think you don't give enough credit to the Pi's GPU. But more importantly, you've committed the sin of premature optimization in that you're predicting performance without actually measuring anything first.

Here, incidentally, are the specs on the new Pi 4's built-in A/V processing power:

Pi 4 Specs wrote:
Video and sound: 2 x micro-HDMI ports supporting 4K@60Hz displays via HDMI 2.0, MIPI DSI display port, MIPI CSI camera port, 4-pole stereo output and composite video port



I have, incidentally, set up video kiosk systems for HDMI displays based on the Pi 3 using the mplayer program. The client was quite happy.

If you want to do transcoding or from-scratch rendering, of course, that's a different matter, and with the right workload you can melt down virtually anything. Still, you can do a lot with a single Pi. And splitting a stream to chew on individual frames and then recombining them is hardly hard work. AVI packets are designed to be helpful in that regard. My biggest problem, in fact, has been in getting the audio synced with the video when working with VOBs. You may not be able to build a Pixar-grade rendering farm with anything less than a room full of Pis, but then again, their own render-farm room is pretty darned big itself.

As to whether one should use the Pi for a render farm, given other options and budgets is open to discussion. You would be even worse off trying to do that kind of work on a top-of-the-line IBM mainframe. GPUs are not very smart, just massively parallel, and while the ones in the Pi's are respectable, there's only one GPU and it's hard-wired to the motherboard.

But rather than just blather on about my own, admittedly limited knowledge and experience, why not look at what happened when someone actually did set up rendering on a Pi cluster. Note that the experiment pre-dates the Pi 4. So improvements can be expected.

https://www.element14.com/community/community/raspberry-pi/raspberrypi_projects/blog/2017/02/06/building-a-render-farm-with-a-bitscope-blade-quatro-running-blender
 
Consider Paul's rocket mass heater.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!