Skip to main content
Topic: Architecture (Read 4060 times)

Architecture

Hi,

just read about Otter a few days ago. I'm stuck with Opera 12.x on Linux (vendor lock-in due to GUI interface, bookmarks, passwords, RSS feeds, ...) so I'm highly interested in the development of Otter.

I've read many tickets/issues on github and several threads in this board. What I'm missing is a general discussion and documentation about the architecture of Otter. (or I still haven't found it, so please excuse me if I missed that one)

As far as I know, Otter should provide most features of Opera + extra. This includes non-web protocols (FTP, IRC, Bittorrent, IMAP(S)/POP3, SMTP, ...), Plugins (Flash, Java, *light, ...) and Extensions (Greasemonkey, Opera Extensions, ...). See also:



On top of that #24 is a beta 2 milestone  :yikes:.

How do you intend to implement these features in the future. Are all non-web-protocols crippled implementations like in Opera? What happens, when a user wants feature X in IRC? Sure Opera is a powerful browser, but most features are not for casual users and not for power users, either. (IRC unusable due to incompatibility with bouncers, Mail without PGP, etc.)

If feature X is implemented in most cases, then we are talking about full-fledged FTP, IRC, XMPP, Mail, SSH, SMB, whatever clients. How does Otter treat this protocol implementations? Should Otter treat these implementations any different than the HTTP(S)/SPDY implementation? What about bugs/crashes? Is it possible that a Bug in the Mail-Client crashes Otter?
What about plugins? What about extensions? Is it possible to modify the behaviour of the IRC-Client?

In my opinion Otter should be a browser in general. Not a "web browser" or a "smb browser" or a "file browser". Just a browser which supports many different protocols. Yes, the focus is HTTP and yes most protocols may be "read only" (like FTP or file://), but the software architecture should support any protocols. Features like bookmarks, history, etc. can be used by any protocol.

What I'm saying is, that Otter should be a rich client platform (something like Eclipse RCP/Netbeans Platform; Sorry I'm mostly a Java and C/Embedded Systems developer :love:), which is robust and can be easily extended.

If I want to add libpurple/IM, how will I do that? Currently everything is pretty much hardcoded, yes? Maybe all of these "modules" should be treated like a dll/so whatever and loaded at run-time? Maybe they should be executed as processes and not as threads to prevent crashes? How are they going to communicate with "Otter"?

I think these are important questions which should be carefully answered, documented (wiki) and implemented (API), before tickets like #24 can be started. Currently the project is young and many features can be implemented on-the-fly, but in a few years a bad architecture is going to kill any new feature requests. 

Re: Architecture

Reply #1
@Tante, sure, documentation would be useful, as when there i lots of code one shouldn't say that code itself is best documentation. ;-)

How do you intend to implement these features in the future. Are all non-web-protocols crippled implementations like in Opera? What happens, when a user wants feature X in IRC? Sure Opera is a powerful browser, but most features are not for casual users and not for power users, either. (IRC unusable due to incompatibility with bouncers, Mail without PGP, etc.)

Level of support will depend on our (users) needs, "core" should have at least basic support for all stuff that is required for web browsing, but also be flexible to allow to extend it when needed (be modular).
I f user wants feature X (not only in IRC) then he or she should ask developers first, if it will make sense then it could be added to core or module. If it doesn't make sense then someone else could always add it to alternative module (for example create more feature rich implementation of IRC client) which could replace existing one (user would need to install it and select as preferred one).
AFAIK libpurple should give as goo enough support for iRC, but it needs more investigation first (so far I know that there exists bindings for Qt, qpurple).
PGP is one of most wanted features for mail, so it is must have, sooner or later. We will try to find suitable library first, no need to reinvent wheel if something useful would already exist (or it's easy to create Qt bindings for it), possibly asking developers of some existing Qt based mail client to help with it (for example by creating version of such application which could run as module for Otter).

If feature X is implemented in most cases, then we are talking about full-fledged FTP, IRC, XMPP, Mail, SSH, SMB, whatever clients. How does Otter treat this protocol implementations? Should Otter treat these implementations any different than the HTTP(S)/SPDY implementation? What about bugs/crashes? Is it possible that a Bug in the Mail-Client crashes Otter?
What about plugins? What about extensions? Is it possible to modify the behaviour of the IRC-Client?

Network protocols are handled by QtWebKit and QtNetwork, that means that improvements for them should be done upstream (so all users of Qt could benefit from them).
For other protocols is varies, some need only basic handlers while others need more complex solutions (like bittorent integrated in transfers manager).
We could try to separate some modules so they wouldn't take down entire application, it should be doable, one way or another.
As mentioned earlier most of modules (if not all) should be replaceable by other implementations.
And yes, extensions are planned (Firefox and Chrome APIs), but it's something scheduled post 1.0 release.

In my opinion Otter should be a browser in general. Not a "web browser" or a "smb browser" or a "file browser". Just a browser which supports many different protocols. Yes, the focus is HTTP and yes most protocols may be "read only" (like FTP or file://), but the software architecture should support any protocols. Features like bookmarks, history, etc. can be used by any protocol.

It' up to be decided, if it makes sense to go the same way as Konqueror did (and ended up being replaced by Dolphin and rekonq...).
We don't want let users think that Otter is bloated or something like that, it's hard to get it right. ;-)

If I want to add libpurple/IM, how will I do that? Currently everything is pretty much hardcoded, yes? Maybe all of these "modules" should be treated like a dll/so whatever and loaded at run-time? Maybe they should be executed as processes and not as threads to prevent crashes? How are they going to communicate with "Otter"?

Yes, currently it's a bit hard-coded (Window object decides which module should be used) but as soon as it will be really needed (well, obviously we have more important stuff to work on right now ;-)) they will become proper modules, managed by ModulesManager and loaded from DLL / SO files (with exception of basic modules, to ensure that basic features will always work).
It will take some time to make it good enough, but sooner or alter we should find some solution for running them separately. For communicating we could use for example QLocalSocket, if nothing better would show up.

I think these are important questions which should be carefully answered, documented (wiki) and implemented (API), before tickets like #24 can be started. Currently the project is young and many features can be implemented on-the-fly, but in a few years a bad architecture is going to kill any new feature requests.

True, although in case of FTP it's a bit different story, what we need to do there is mostly just to parse output from FTP server, using deprecated QFtp class as reference.
Keep in mind that we are using quite powerful framework (Qt) which implements most of basic stuff, usually in enough flexible form to let to extend it if needed.
Nadszedł już czas, najwyższy czas, nienawiść zniszczyć w sobie.
The time has come, the high time, to destroy hatred in oneself.

Re: Architecture

Reply #2

@Tante, sure, documentation would be useful, as when there i lots of code one shouldn't say that code itself is best documentation. ;-)

As long as it doesn't stay that way. E.g. Qt isn't as successful as it is, just because it's technically good. It's successful, because it's well documented.

It' up to be decided, if it makes sense to go the same way as Konqueror did (and ended up being replaced by Dolphin and rekonq...).

Konqueror was always too much integrated in KDE and focused more on the backend, than on the user interface (KIO, KHTML, ...). AFAIK dolphin still uses many parts of Konqueror (KIO) and attacked the faulty UI of Konqueror. I think Otter is more like Dolphin, which uses existing software to create a good user experience. (instead of developing highly developed backends like Konqueror did. Also I have to admit I don't know how much time has been used to develop the UI and how much time the backends)
And does anyone use rekonq seriously?

We don't want let users think that Otter is bloated or something like that, it's hard to get it right. ;-)

What does "bloated" mean? Performance? Users will always say that more features require more CPU, memory whatever, even though these features can be completely disabled (modules not loaded). Used developer time, which could be used to develop "more important" parts? I think users will only complain, if features are crippled, like in opera (there were many discussions in the opera forums in the past). The advantage of Otter is, that it's open source and attracts more and more developers. These developers must be able to easily extend and maintain Otter, like adding a SFTP client (different user interface, different logic, but same goal as FTP, so there is a high probability that such a request arises). These features should be, which might be delivered by a 3th party, very good encapsulated. Maybe something like the linux kernel does with modules.

Just don't wait too long, until you develop a proper "Otter Module System", especially if you want a RSS reader in beta 3.

Re: Architecture

Reply #3

And does anyone use rekonq seriously?

I use it unseriously. I have strong historical attraction for Konqueror, but it's hard to make one's main browser something that only jibes properly with one single desktop environment. This is the problem with both Konqueror and Rekonq.


We don't want let users think that Otter is bloated or something like that, it's hard to get it right. ;-)

What does "bloated" mean?

I think Emdek meant precisely that "bloated" is a matter of perception from many highly debatable angles, but he'd still wish to escape the accusation of bloatedness as far as possible. Even though, yeah, it's not possible. For some people feature-richness is bloat, while for others sluggish behaviour on limited resources is bloat, regardless of the feature set. I lean towards the latter definition.

Careless coding (multi-process architecture promotes disregard for system resources!) and unconsidered architecture are bloat to me, but the more popular definition is probably the former. The multitude of features in a confounding default interface made many people call Opera Presto bloated.

A good thread you started. I like these philosophical topics :)

Re: Architecture

Reply #4

Careless coding (multi-process architecture promotes disregard for system resources!)

I couldn't agree more. Which provokes a question: once  Otter adopts QtWebEngine (i.e. Chromium) instead of QtWebkit, it will necessarily become multi-process? Or not?

On my old computer Chrome is sluggish, heavy on RAM, while Opera is not the best in terms of responsiveness, but for sure the least resource consuming. I know it is up for discussion, but I think single-process browsers are better.

Re: Architecture

Reply #5
Konqueror was always too much integrated in KDE and focused more on the backend, than on the user interface (KIO, KHTML, ...). AFAIK dolphin still uses many parts of Konqueror (KIO) and attacked the faulty UI of Konqueror. I think Otter is more like Dolphin, which uses existing software to create a good user experience. (instead of developing highly developed backends like Konqueror did. Also I have to admit I don't know how much time has been used to develop the UI and how much time the backends)

Well, the main difference for me was that UI of Konqueror seemed less modern to end user, also they couldn't focus on one part (like Dolphin focused on browsing files).
And yes, they share kparts but Dolphin allows only for browsing files and directories, while Konqueror used to embed almost any content (I guess that it had to do some exceptions, so it wouldn't open KATE kpart when attempting to open plain text files ;-)).
I guess that user facing parts got a more attention (as long as we include interfaces for stuff like history, cookies, cache, transfers etc.), as QtWebKit requires only "few" methods to be reimplemented plus extending network stack (QNetworkAccessManager integration, something that QtWebEngine is missing so far...), which allows us to handle cookies, user agent spoofing, collect stats (I really love advanced progress bar ;-)), modify headers and will allow to handle content blocking (that part where we block entire pages).

What does "bloated" mean?

Just like ersi said, everyone defines it differently, I would like to avoid provoking them as much as possible. ;-)

Uhm, if we want to support SSH and stuff like that we should consider adding separate FTP module (while having basic, read-only stuff built-in) which could be invoked when needed.

Just don't wait too long, until you develop a proper "Otter Module System", especially if you want a RSS reader in beta 3

Well, API is already there, it might need changes in future but modules are already separated, having base classes to extend (ContentsWidget, WebWidget and WebBackend), but so far they all end up in single binary.

Which provokes a question: once  Otter adopts QtWebEngine (i.e. Chromium) instead of QtWebkit, it will necessarily become multi-process? Or not?

We will support multiple backends (at least QtWebKit and QtWebEngine, it's also possible to have basic Gecko backend at least under Linux), while QtWebEngine will have to be multi-process we will still allow to switch to QtWebKit (which will be available at least until Qt6, which gives many years of upstream support) or other backends, if they will materialize.
Nadszedł już czas, najwyższy czas, nienawiść zniszczyć w sobie.
The time has come, the high time, to destroy hatred in oneself.

Re: Architecture

Reply #6
For some people feature-richness is bloat, while for others sluggish behaviour on limited resources is bloat, regardless of the feature set. I lean towards the latter definition.

I imagine most (former) Opera users lean toward the latter definition.

Re: Architecture

Reply #7
I think, a good architecture for Otter would be a modular one with local servers to which commands can be sent, similar to X on Unix systems. As far as I understand it, X usually runs as a local server on a Unix workstation and programs written to use X for graphical output use it to display their user interface. As I understand Otter, it uses engines that can draw formatted web pages from HTML files and displays the result in a prominent area of its program window. Since the decorations, that Otter places around the naked rendering area, might not be ideal for every environment, it would be nice to separate the control elements from the canvas. From what I've read, this is already useful for full-screen mode. It could also be useful for different user interfaces, for instance one optimized for touch screens.

Now, to protect against accusations of being bloated, how about presenting Otter as simply a lean interface to control those specialized engines, that people would like to be part of their browser? Ideally, this lean interface could be freely configured in its visual characteristcs by the users themselves. The control interface should send messages to those components, that provide the actual functionality, via some communication facility. Sockets seem to be a standard abstraction that components can use to connect to each other and then communicate via some protocol. Network sockets in particular would allow to send commands from anywhere. It would allow, for instance, to have a control interface on your smartphone and the rendering canvas on your monitor. I don't know, if people would use this to watch web videos from their couch, but it could also be used for other things like adding torrents to your Otter download component running on your computer at home from some other place where you happen to be. Also, Otter would not be bound to one user interface. Adapting Otter to the holographic 3D interfaces, that might be in fashion ten years from now, would be much simpler, if the components, that Otter's current user interface controls, could be controlled through messages rather than called through hard-coded mappings of buttons to procedures.

P.S.: Thinking about it, X on Unix systems might not have been the most representative thing to illustrate the concept. I hope I could make clear what I meant with the subsequent explanations.

Re: Architecture

Reply #8
@Somebody, I'm afraid that separating it that much would complicate code too much, also eliminating support for some less used platforms...
We could for sure try a bit different model, where GUI is run in single thread (as usual, in Qt) but some modules (without forcing all of them) could run their backends separately, using sockets for communication.
Such model should be good enough in most cases while not introducing platform dependent solutions (AFAIK Chrome uses platform specific solutions to glue all tabs into single window).
Nadszedł już czas, najwyższy czas, nienawiść zniszczyć w sobie.
The time has come, the high time, to destroy hatred in oneself.