Rugby
The future of Rugby
07-Nov-2007 20:48
Any blog posts that start with "The future of..."
could sound a bit frightening. In this case, it might
even be true (gasp!). With R3 storming ahead and
REBOL/Services proving itself worthy through DevBase,
I'm not sure what place Rugby has at this time.
Furthermore Rugby will definitely not be compatible with R3 as the entire port handling system has changed and HTTP scheme has been rewritten from scratch.
The original author, Maarten Coopmans, has chosen not to pursue further development of it.
Please note that I will of course continue to serve the sources for Rugby. They will not go away. Some annoying bugs still remain, and one of these bugs is a deep one that occurs in R2 ports which causes R2 to hang on sending information from the Rugby client to the Rugby server in very specific cases. I've not had the time or the inclination to fix this bug, as it's very difficult to track down. There are also problems with sending words and objects across a Rugby connection and I've not managed to fix those properly either. These things can be fixed in one place and then cause trouble in another place.
Rugby is very nice for quick RPC systems, but it scales only so much, so use it if you want some simple single-threaded RPC stuff. For more complex high-performance multi-threaded fancy networking, wait for R3.
Furthermore Rugby will definitely not be compatible with R3 as the entire port handling system has changed and HTTP scheme has been rewritten from scratch.
The original author, Maarten Coopmans, has chosen not to pursue further development of it.
Please note that I will of course continue to serve the sources for Rugby. They will not go away. Some annoying bugs still remain, and one of these bugs is a deep one that occurs in R2 ports which causes R2 to hang on sending information from the Rugby client to the Rugby server in very specific cases. I've not had the time or the inclination to fix this bug, as it's very difficult to track down. There are also problems with sending words and objects across a Rugby connection and I've not managed to fix those properly either. These things can be fixed in one place and then cause trouble in another place.
Rugby is very nice for quick RPC systems, but it scales only so much, so use it if you want some simple single-threaded RPC stuff. For more complex high-performance multi-threaded fancy networking, wait for R3.
|
Even More Rugby Fixes
08-Apr-2007 13:43
The first
bug fix went not as deep as I thought it
should. The fix made it possible to evaluate
Rugby shared functions where the first argument
is a word.
Let's say that we have a Rugby shared function test-func in a net context and it returns the first argument:
But having multiple arguments where just one of them is a word, caused the same bug to appear again. test-func2 here accepts two arguments:
The new fix, fixes this bug by asking whether the argument is of type any-word and converts it to a lit-word! before evaluating the function server side.
This happens inside Rugby's function to parse and collect function refinements and arguments. The change looks like this:
Before:
After:
I'm still troubled that this is not a truly elegant solution to the problem, treating the symptoms like this rather than going for the cause. It works for now though, and you can download the latest version 5.0.4 here.
The source file now also contains a history of changes made by me.
Let's say that we have a Rugby shared function test-func in a net context and it returns the first argument:
net/test-func 'a
== a
But having multiple arguments where just one of them is a word, caused the same bug to appear again. test-func2 here accepts two arguments:
net/test-func2 5 'a
** Script Error: a has no value
The new fix, fixes this bug by asking whether the argument is of type any-word and converts it to a lit-word! before evaluating the function server side.
This happens inside Rugby's function to parse and collect function refinements and arguments. The change looks like this:
Before:
p2: [set w word!
(if ref-mode [append/only statement get bind to-word w 'comm])]
After:
p2: [set w word!
(if ref-mode [
w: get bind to-word w 'comm
append/only statement either any-word? w [to lit-word! w][w]]
)
]
I'm still troubled that this is not a truly elegant solution to the problem, treating the symptoms like this rather than going for the cause. It works for now though, and you can download the latest version 5.0.4 here.
The source file now also contains a history of changes made by me.
More Rugby Fixes
03-Apr-2007 18:15
I finally found the "mirror" of a bug to an older
MOLD bug, which prevented sending objects unharmed
(e.g. properly MOLD'ed) from the server to the
client. The mirror of that bug would not let objects
go through from the client to the server, but that
works now by changing the MOLD to MOLD/ALL in
COMPOSE-MSG on both server and client.
CHECK-MSG in both the client and the server part of Rugby were broken and did not calculate the checksum of the message according to how the checksum is built in COMPOSE-MSG. This has also been fixed.
You can get the latest rugby.r here.
To compare, the older version without this fix is here.
CHECK-MSG in both the client and the server part of Rugby were broken and did not calculate the checksum of the message according to how the checksum is built in COMPOSE-MSG. This has also been fixed.
You can get the latest rugby.r here.
To compare, the older version without this fix is here.
Returning any-word! with Rugby
20-Mar-2007 13:44
I discovered that Rugby evaluates anything before
it's returned, so if you are returning any-word!
values (words, lit-words, etc.), like this:
test: does ['some-word]
Rugby will attempt to evaluate some-word.
If it fails, because it's unset, Rugby returns an error in an internal function. I believe this to be incorrect, because this breaks the paradigm of having Rugby be able to serve any REBOL function unchanged.
I've not had this behavior verified as correct by Maarten Koopmans, but I made a patch to the to-result that passes any-word! values through, while other return values are still evaluated.If you are using code, that relies on
the word to be evaluated before being returned by
Rugby, your code will break.
UPDATE:
An improved fix checks for set values in a returned word, so if your word has a value, it will be evaluated. Words that don't have a value, as well as lit-words will still not be evaluated. It has been approved by Maarten as a fix.
I've uploaded a patched version here.
test: does ['some-word]
Rugby will attempt to evaluate some-word.
If it fails, because it's unset, Rugby returns an error in an internal function. I believe this to be incorrect, because this breaks the paradigm of having Rugby be able to serve any REBOL function unchanged.
I've not had this behavior verified as correct by Maarten Koopmans, but I made a patch to the to-result that passes any-word! values through, while other return values are still evaluated.
UPDATE:
An improved fix checks for set values in a returned word, so if your word has a value, it will be evaluated. Words that don't have a value, as well as lit-words will still not be evaluated. It has been approved by Maarten as a fix.
I've uploaded a patched version here.
Rugby Optimizations Update
15-Feb-2007 20:49
Now I've had a little time to do some testing and
built a set of test cases for server and client.
These will be uploaded soon, because I want to test
for some more bugs in Rugby.
It turns out that when the server reads 4096 bytes at a time from the buffer, it still hangs after 8680 bytes and then every 16 bytes above that. Therefore I lowered the read size to 2048 bytes which seems to work (purely by guessing unfortunately).
It's nearly as fast, but more importantly, it seems to be stable so far up to sending 50000 bytes from the client to the server.
A bug that causes a hang and 100% cpu on the server when sending exactly 3810 bytes to the server has been fixed.
Remember that the old sources are still available at the original location in the downloads section. The patched version is still available here.
It turns out that when the server reads 4096 bytes at a time from the buffer, it still hangs after 8680 bytes and then every 16 bytes above that. Therefore I lowered the read size to 2048 bytes which seems to work (purely by guessing unfortunately).
It's nearly as fast, but more importantly, it seems to be stable so far up to sending 50000 bytes from the client to the server.
A bug that causes a hang and 100% cpu on the server when sending exactly 3810 bytes to the server has been fixed.
Remember that the old sources are still available at the original location in the downloads section. The patched version is still available here.
Rugby Optimizations
15-Feb-2007 14:23
I found that sending a fairly large amount of data
from a client to the server takes an enormous amount
of time while eating 100% CPU time. After a bit of
digging, it turns out that Rugby reads data from the
client port 16 bytes at a time, once the data has
been transferred into the buffer from the client.
This gives horrendous performance on even small files at 10-50 kb in size, taking several seconds for Rugby to finish reading from the port. Furthermore, a function to detect EOD was performing a FIND on the incoming data on each 16 byte read. As this data grows, the FIND operation slows down a lot.
I patched the source to rugby.r to reduce the size of how much data is fed to the FIND function to 50 bytes of data. Then I also use adaptive resizing of how much data is read from the port, starting out at 4096 bytes from after reading Content-Length: (apparently the maximum allowed size) and then scaling down to 16 bytes when nearing the end.
A small test which took 14 seconds to complete on about 170 kb of data on the original rugby.r, now clocks in at 0.2 to 0.3 seconds!
But there is a small catch: The 16 byte alignment seems to be necessary to read, when getting near the end of data in the buffer. If this alignment isn't there, EOD isn't detected and Rugby hangs. I can presently not be 100% sure that the 16 byte alignment is reached every time with the adaptive resizing, so please test the code well.
This gives horrendous performance on even small files at 10-50 kb in size, taking several seconds for Rugby to finish reading from the port. Furthermore, a function to detect EOD was performing a FIND on the incoming data on each 16 byte read. As this data grows, the FIND operation slows down a lot.
I patched the source to rugby.r to reduce the size of how much data is fed to the FIND function to 50 bytes of data. Then I also use adaptive resizing of how much data is read from the port, starting out at 4096 bytes from after reading Content-Length: (apparently the maximum allowed size) and then scaling down to 16 bytes when nearing the end.
A small test which took 14 seconds to complete on about 170 kb of data on the original rugby.r, now clocks in at 0.2 to 0.3 seconds!
But there is a small catch: The 16 byte alignment seems to be necessary to read, when getting near the end of data in the buffer. If this alignment isn't there, EOD isn't detected and Rugby hangs. I can presently not be 100% sure that the 16 byte alignment is reached every time with the adaptive resizing, so please test the code well.
Hosting Rugby Sources
19-Jan-2007 05:36
I have been allowed, after agreement with Mr. Maarten
Koopmans, to host the sources to Rugby, the very
excellent RPC system for REBOL. Information on how to
use Rugby is too scattered and old, so I want to
change that! There will hopefully be brand new
documentation available soon.
You can find the sources to the latest version, called Rugby-X, in the Downloads area.
You can find the sources to the latest version, called Rugby-X, in the Downloads area.