Forum Moderators: coopster

Message Too Old, No Replies

Script with very heavy text processing tasks

         

TuanLa1972

7:36 am on Jul 1, 2011 (gmt 0)

10+ Year Member



Hi there.
I need help for script with very heavy text processing tasks.

The script takes a text input from users that can be 500 words long, processes it and sends back.

Recently, for 500 words text my script needs about 2 minutes with 100 CPU usage.

I've tried to optimize as much as I can, it doesn't help much.

I am thinking about distributed system that the main script sends each line of text to different minor scripts, hosted some where else for processing. Main script collects results and merges final text then.

This is my very basic idea, but I don't know how to do that with php.

Please, give me your recommendations.

Thanks a lot in advance.

rocknbil

3:46 pm on Jul 1, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



500 or even 5000 words is not really very heavy. If you're taking 2 minutes/100% to process 500 words, you have to be doing something wrong, can we look at your current script? Try to cut it down to just the relevant portions, huge code dumps are often ignored.

penders

10:51 pm on Jul 1, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yeah that's what I was thinking, but then may be I thought this was some kind of enigma machine?!

(@rocknbil - Congrats on the 7k posts!)

Leosghost

11:02 pm on Jul 1, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Ditto penders #:4333913 especially congrats rocknbil :)

rocknbil

4:38 pm on Jul 5, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Haha . . thank you, I never would have noticed . . .

Now how about a peek at that code?