Netfow on a grand scale
Hi!
We are hosting some big websites in our data centre with two GigaBit Uplinks to Internet.
A Cisco 6503 (only switching) speaks Netflow (layer 2) to a Dell 1860 (Dual XEON with 4GB RAM) with Debian Sarge on it.
We are confronted with 'event-times' and 'non-event-times'.
On non-event-times we have a traffic noise about 20-40MBit/s. (about 2000 flows/s)
On event-times we experience traffic over 1GBit/s (estimated 40.000 flows/s and more, damn much - I know
).
Till today we used a Dell 850 (2.8Ghz) with 1GB RAM, and with traffic about 100MBit/s
the machine was completely overloaded. (Netflow said that switch/router time is not synchonized, load was about 10)
If I am right sampled netflow dont't help me, because it only decreases performance used by the router.
And netflow aggregation is not supported by Netflow Analyzer because of missing version 8 support of netflow.
Now the new Dell 1860 (with traffic about 40MBit/s and about 2000 flows) although it's a bigger machine,
runs at a load of 2. So I guess with the next event/s (which are actually the soccer-worldcup-games) my
new server is also totally overloaded.
I allready did the 'database-optimizing" to the accordant amount of RAM
(--key_buffer_size=720000000 --innodb_buffer_pool_size=848000000), but with no improvement.
What else can I do to account all traffic? Is there another possibility for optimizing the database?
Installed RAM is completly used by java, CPU is nearly idle.
thank you for help!
best regards
Bernhard
We are hosting some big websites in our data centre with two GigaBit Uplinks to Internet.
A Cisco 6503 (only switching) speaks Netflow (layer 2) to a Dell 1860 (Dual XEON with 4GB RAM) with Debian Sarge on it.
We are confronted with 'event-times' and 'non-event-times'.
On non-event-times we have a traffic noise about 20-40MBit/s. (about 2000 flows/s)
On event-times we experience traffic over 1GBit/s (estimated 40.000 flows/s and more, damn much - I know
).Till today we used a Dell 850 (2.8Ghz) with 1GB RAM, and with traffic about 100MBit/s
the machine was completely overloaded. (Netflow said that switch/router time is not synchonized, load was about 10)
If I am right sampled netflow dont't help me, because it only decreases performance used by the router.
And netflow aggregation is not supported by Netflow Analyzer because of missing version 8 support of netflow.
Now the new Dell 1860 (with traffic about 40MBit/s and about 2000 flows) although it's a bigger machine,
runs at a load of 2. So I guess with the next event/s (which are actually the soccer-worldcup-games) my
new server is also totally overloaded.
I allready did the 'database-optimizing" to the accordant amount of RAM
(--key_buffer_size=720000000 --innodb_buffer_pool_size=848000000), but with no improvement.
What else can I do to account all traffic? Is there another possibility for optimizing the database?
Installed RAM is completly used by java, CPU is nearly idle.
thank you for help!
best regards
Bernhard