AI cloud...

[RichD]

An item in WSJ about Coreweave. They operate
a GPU \'cloud server\', or whatever, soaking up
as many Nvidia GPU as they can get, for AI applications.

I know near zip about this business. Like what
makes Nvidia chips ultra optimized for AI? And
what\'s inside the Coreweave boxes, are they building
custom parallel computers?

And what\'s their business model? I mean, what are
they selling, who are the customers, what do those
customers get for their money, that they can\'t do
with off the shelf Pentium boards?

I post this here, hoping somebody has some inside
poop on this topic -

--
Rich

 

[Martin Brown]

On 26/08/2023 22:47, RichD wrote:

An item in WSJ about Coreweave. They operate
a GPU \'cloud server\', or whatever, soaking up
as many Nvidia GPU as they can get, for AI applications.

I know near zip about this business. Like what
makes Nvidia chips ultra optimized for AI? And
what\'s inside the Coreweave boxes, are they building
custom parallel computers?

As much as anything sheer coincidence that massively parallel hardware
intended for graphics texture mapping and convolutions happens to be
almost exactly what the deep neural net AI codes need to be super fast.

This has some PR fluff in as well but isn\'t a bad introduction:

https://blogs.nvidia.com/blog/2016/01/12/accelerating-ai-artificial-intelligence-gpus/

Nvidia can\'t believe their luck! Their GPUs are a way to do this sort of
AI with much less power used than in conventional CPUs. They may get out
evolved by IBM\'s analogue memristors based method - only time will tell.

https://research.ibm.com/blog/artificial-memtransistive-synapse

And what\'s their business model? I mean, what are
they selling, who are the customers, what do those
customers get for their money, that they can\'t do
with off the shelf Pentium boards?

Same as most of the modern chess engines using AI - they offload the
deep AI neural net onto the graphics subsystem to take advantage of its
massively parallel hardware. Scientific computing has been doing it for
much longer than AI with specialised libraries and some pain and
suffering. It works best on certain types of replicated code.

I post this here, hoping somebody has some inside
poop on this topic -

I\'m not directly involved in the AI side, but some of the toolchain I
use is intended to facilitate that sort of stuff. Annoying side effect
is that some related compiler tools will refuse to install if your
graphics card isn\'t sufficiently powerful!

--
Martin Brown

 

[Don Y]

On 8/27/2023 1:13 AM, Martin Brown wrote:

As much as anything sheer coincidence that massively parallel hardware intended
for graphics texture mapping and convolutions happens to be almost exactly what
the deep neural net AI codes need to be super fast.

Perceptrons are trivial to implement:
output = decide( sum( weight * input ) + bias )
where \"decide()\" is a function that maps the accumulated weighted
inputs into the output range.

For a single perceptron, you effectively get a linear
equation that makes an assessment of JUST the N inputs
available to it:
will_I_like_it = examine(sweet, salty, spicey, hot, cold)
where the result (decision) is a simple weighted sum of the
influences of these various inputs on that one decision.

Normally, you\'ll have many perceptrons operating on the same
input\'s so these computations can be done in parallel.

The outputs from this \"row\" of perceptrons will act as *inputs*
to feed another row of perceptrons -- so another group of
parallel operations.

Eventually, you get to the ultimate output perceptrons (fed by
a row of \"hidden\" perceptrons), again computed in parallel,
which makes available the network\'s \"decisions\".

The number of such \"hidden\" rows of perceptrons is indicative
of the AI\'s \"depth\"; more hidden neural interrelationships.

Of course, if you have as many processors as TOTAL perceptrons,
you can start processing the next set of inputs while the
previous set are making their way through the successive
layers of hidden perceptrons.

Training the network involves applying inputs for which the
(ultimate) output(s) are known and then tweeking the
numerous weights until the entirety of the network
comes up with the same predicted output as you have already
*observed* in your training data.

Doing this on a sequential processor is obviously very tedious:

for each layer of perceptrons {
for each perceptron in that layer {
for each input to that perceptron {
sum := sum + input * weight
}
output = decide(sum + bias)
}
}

Imagine each iterated variable being tens/hundreds/thousands of
instances to get a feel for how long the serialized version takes!

E.g., instead of having just one ultimate output (\"will I like it\"),
there will often be many. E.g., to recognize gestures, I have an
output neuron for each possible gesture: 0, 1, 2, 3, ... 9,
YES, NO, UP, DOWN, LEFT, RIGHT, UNDO, CANCEL, REPEAT, ...

Storing the weights is the equivalent of storing the \"knowledge\"
embodied in the network AS OF THAT POINT IN TIME (e.g., so I
can train a gesture recognizer to recognize MY gesturing style
and then replace the weights with those that the network had
learned for YOUR gesturing style) cuz you can continue to train
it OR reset the weights to defaults and start anew)

 

[Don Y]

On 8/27/2023 2:25 AM, Don Y wrote:

On 8/27/2023 1:13 AM, Martin Brown wrote:
As much as anything sheer coincidence that massively parallel hardware
intended for graphics texture mapping and convolutions happens to be almost
exactly what the deep neural net AI codes need to be super fast.

Note that you can buy nvidia GPUs that don\'t even have the video
(display) hardware. I have several teslas, here, looking for
a use... :>

 

[John Larkin]

On Sat, 26 Aug 2023 14:47:25 -0700 (PDT), RichD
<r_delaney2001@yahoo.com> wrote:

An item in WSJ about Coreweave. They operate
a GPU \'cloud server\', or whatever, soaking up
as many Nvidia GPU as they can get, for AI applications.

I know near zip about this business. Like what
makes Nvidia chips ultra optimized for AI? And
what\'s inside the Coreweave boxes, are they building
custom parallel computers?

And what\'s their business model? I mean, what are
they selling, who are the customers, what do those
customers get for their money, that they can\'t do
with off the shelf Pentium boards?

I post this here, hoping somebody has some inside
poop on this topic -

What is AI anyhow? I read about somebody raising $20 million and
hiring some programmers to start an AI company. What does that even
mean? Is there some theoretical basis for the current AI boom, or do
people just use the term for writing a bunch of code?

Neural Network theory, the cartoon version of a real brain, has been
around for over a century.

 

[Anthony William Sloman]

On Sunday, August 27, 2023 at 11:57:52 PM UTC+10, John Larkin wrote:

On Sat, 26 Aug 2023 14:47:25 -0700 (PDT), RichD <r_dela...@yahoo.com> wrote:

An item in WSJ about Coreweave. They operate a GPU \'cloud server\', or whatever, soaking up as many Nvidia GPU as they can get, for AI applications.

I know near zip about this business. Like what makes Nvidia chips ultra optimized for AI? And what\'s inside the Coreweave boxes, are they building custom parallel computers?

And what\'s their business model? I mean, what are they selling, who are the customers, what do those customers get for their money, that they can\'t do with off the shelf Pentium boards?

I post this here, hoping somebody has some inside poop on this topic -

What is AI anyhow? I read about somebody raising $20 million and hiring some programmers to start an AI company. What does that even mean? Is there some theoretical basis for the current AI boom, or do people just use the term for writing a bunch of code?

The most recent AI buzz was about the Alpha Fold program for predicting protein structure from the protein amino acid sequence.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9580946/

https://en.wikipedia.org/wiki/Google_DeepMind

It\'s all about programs that can absorb more information than a human brain seems to be able to hold and make some kind of sense of it. The large language models that Google Translate exploits seem to work in the same kind of way. For some problems 86 billion neurones don\'t seem to be enough.

> Neural Network theory, the cartoon version of a real brain, has been around for over a century.

But getting the neural networks (or whatever they use) big enough to be useful seems to be the real trick.

--
Bill Sloman, Sydney

 

[Martin Brown]

On 27/08/2023 14:57, John Larkin wrote:

On Sat, 26 Aug 2023 14:47:25 -0700 (PDT), RichD
r_delaney2001@yahoo.com> wrote:

An item in WSJ about Coreweave. They operate
a GPU \'cloud server\', or whatever, soaking up
as many Nvidia GPU as they can get, for AI applications.

I know near zip about this business. Like what
makes Nvidia chips ultra optimized for AI? And
what\'s inside the Coreweave boxes, are they building
custom parallel computers?

And what\'s their business model? I mean, what are
they selling, who are the customers, what do those
customers get for their money, that they can\'t do
with off the shelf Pentium boards?

I post this here, hoping somebody has some inside
poop on this topic -

What is AI anyhow? I read about somebody raising $20 million and
hiring some programmers to start an AI company. What does that even
mean? Is there some theoretical basis for the current AI boom, or do
people just use the term for writing a bunch of code?

The code is mostly already written and the skill is in training it to do
something useful in the real world. Googles AlphaGo Zero learnt the game
from just the basic rules to global supremacy in about 3 days. It also
surpassed the best humans in that timescale and the previous computer
record holder. It found new defensive patterns that had never arisen in
human play in a couple of millennia. That is non-trivial.

https://en.wikipedia.org/wiki/AlphaGo_Zero

Go is to Chess as Chess is to Checkers (Draughts in UK)

Neural Network theory, the cartoon version of a real brain, has been
around for over a century.

It is a pretty good model for the way the brain works. The only snag is
modifying the huge number of interconnections and nodes needed.

It comes and goes in waves. First proposed in 1944 by Warren McCullough
and Walter Pitts at MIT

https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414

Currently back in fashion because it fits well with the available high
performance hardware that high end gaming graphics cards in particular
have unleashed at unprecedented MIPS per dollar.

When I was in Japan it was in vogue as \"Fuzzy Logic\" which covered a
multitude of sins but meant that Shinkansen trains always stopped with
their doors exactly in front of marked passenger queues every time!

--
Martin Brown

 

[Don Y]

On 8/27/2023 7:59 AM, Martin Brown wrote:

It is a pretty good model for the way the brain works. The only snag is
modifying the huge number of interconnections and nodes needed.

The human brain typically approaches problems in several phases/levels.
Knowing what to look at to make a decision is more than half of the problem.
If you look at EVERYTHING, you quickly exceed the ability of even the most
\"resource intensive\" AIs to address the problem -- in any sort of reasonable
amount of time.

So, the design of the input neurons becomes a big factor in the efficiency
and effectiveness of the AI as it gives you a clear opportunity to reduce
the data presented to the network.

AIs win over humans because there is comparatively little cost to dragging
in extra observations that *may* be pertinent; a human would not want to
consider something unless he had a reasonably good idea (hunch) that it
*may* be important. Humans get fatigued and make mistakes when chewing on
a particular problem for long periods.

So, an AI can make diagnoses that a human wouldn\'t because it can consider
more things that (presently) aren\'t considered significant in a particular
condition diagnosis.

It comes and goes in waves. First proposed in 1944 by Warren McCullough and
Walter Pitts at MIT

https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414

Currently back in fashion because it fits well with the available high
performance hardware that high end gaming graphics cards in particular have
unleashed at unprecedented MIPS per dollar.

Almost all of these \"advanced technologies\" rely on inexpensive, commonplace
hardware to tackle the tasks presented. Running even a small ANN on something
as fancy as a (40+ year old VAX) would just be an academic exercise.

It\'s also appropriate, now, because we are much more accustomed to machines
doing things autonomously. We now regard having to TELL them what to do as
a chore that should be able to benefit from technology as much as their
*doing* those things has, thus far.

And, we now think it realistic to consider having a machine REtrain itself
when its operations start to fall short of our expectations (instead of
having to replace/upgrade the device).

When I was in Japan it was in vogue as \"Fuzzy Logic\" which covered a multitude
of sins but meant that Shinkansen trains always stopped with their doors
exactly in front of marked passenger queues every time!

 

[RichD]

On August 27, John Larkin wrote:

An item in WSJ about Coreweave. They operate
a GPU \'cloud server\', or whatever, soaking up
as many Nvidia GPU as they can get, for AI applications.
I know near zip about this business. Like what
makes Nvidia chips ultra optimized for AI? And
what\'s inside the Coreweave boxes, are they building
custom parallel computers?
And what\'s their business model? I mean, what are
they selling, who are the customers,

What is AI anyhow? I read about somebody raising $20 million and
hiring some programmers to start an AI company. What does that even
mean? Is there some theoretical basis for the current AI boom, or do
people just use the term for writing a bunch of code?

The VC industry is built on home runs, not singles and
stolen bases. They\'re betting/hoping that another Facebook will
pop up. That pays for the strikeouts.

Neural Network theory, the cartoon version of a real brain, has been
around for over a century.

(almost) true, which means much of the hype is just hype.

What happened is that processors got faster, storage got
cheaper, and fiber optics was laid everywhere. These factors
combined to enable the training of big neural nets.

By the way, the same could be said of Fazebook. Those same
3 factors matured in 2005, when the company started.

Everybody makes a big splash about the coders, but it
was the hardware, the plumbing, which created Fazebook.
There were similar attempts before 2000, but graphics and
communications were too slow, so they fizzled.

--
Rich

 

[RichD]

On August 27, Martin Brown wrote:

An item in WSJ about Coreweave. They operate
a GPU \'cloud server\', or whatever, soaking up
as many Nvidia GPU as they can get, for AI applications.
I know near zip about this business. Like what
makes Nvidia chips ultra optimized for AI? And
what\'s inside the Coreweave boxes, are they building
custom parallel computers?

As much as anything sheer coincidence that massively parallel hardware
intended for graphics texture mapping and convolutions happens to be
almost exactly what the deep neural net AI codes need to be super fast.
https://blogs.nvidia.com/blog/2016/01/12/accelerating-ai-artificial-intelligence-gpus/
Nvidia can\'t believe their luck! Their GPUs are a way to do this sort of
AI with much less power used than in conventional CPUs.

And what\'s their business model? I mean, what are
they selling, who are the customers, what do those
customers get for their money, that they can\'t do
with off the shelf Pentium boards?

Same as most of the modern chess engines using AI - they offload the
deep AI neural net onto the graphics subsystem to take advantage of its
massively parallel hardware. Scientific computing has been doing it for
much longer than AI with specialised libraries and some pain and
suffering. It works best on certain types of replicated code.

Let\'s see... GPU is designed for graphics, i.e. rotations and scaling,
which uses lots of matrix computations. And it so happens that
neural net training requires much the same.

AI developers transfer their datasets to Corewave, and rent time on
their processors, which run 10x faster than Dell. Sounds like the old
IBM mainframe days... they were called \'\'jobs\"

As an aside, the quantum computer developers are planning
the same business model.

I\'m not directly involved in the AI side, but some of the toolchain I
use is intended to facilitate that sort of stuff.

Explicate?

--
Rich