Understanding Dubbo Protocols: A Closer Look at Dubbo’s HTTP Protocol and Its Evolution

Illustrator: FeelThe content should incorporate “Dubbo protocols” effectively.

The sun is blazing red, the flowers are colorful, and hello to all the readers. It’s time to share some insights about Dubbo again. When mentioning the protocols supported by the Dubbo framework, what is your first thought? Perhaps you think of the default dubbo protocol supported by Dubbo, the oft-discussed rest protocol contributed by Dangdang to Dubbo, or today’s focus, http. As of now, the latest version of Dubbo has evolved to 2.7.3, and it supports various protocols: dubbo, hessain, http, injvm, jsonrpc, memcached, native-thrift, thrift, redis, rest, rmi, webservice, xml, among others. Some usage of the protocols hasn’t yet been fully documented in the official documentation. Did the sheer number of supported protocols surprise you?

With so many RPC protocols, some may have questions like: Aren’t rest, jsonrpc, and webservice reliant on http communication? Why is there a standalone http protocol? Before answering this, let’s introduce today’s topic by first discussing what the so-called http protocol is in the Dubbo framework.

HTTP Protocol in Dubbo

Using the http protocol in Dubbo is essentially the same as using any other protocol; you simply specify the protocol.

Code language: javascriptCopy

The server attribute optional values are: jetty, tomcat, servlet.

Once configured, when a service consumer initiates a call to a service provider, standard http protocol communication will occur at the underlying layer. You can find official examples in https://github.com/apache/dubbo-samples, where the submodule dubbo-samples-http builds an example call using the http protocol.

To avoid any misunderstanding, it should be declared here: Throughout this article, “http protocol” specifically refers to the http protocol within Dubbo, not the widely known general http protocol.

Underlying Principles of the HTTP Protocol

Switching from the default dubbo protocol to the http protocol is very straightforward. That’s all there is from the user’s perspective. Next, we will explore its underlying implementation principles.

When reviewing Dubbo’s source code and locating the implementation of HttpProtocol, you may be surprised to find that the http protocol is almost solely implemented with just the HttpProtocol class.

>

By comparison, implementing a custom dubbo protocol involves nearly 30 classes! The relative simplicity of the http protocol implementation can be attributed primarily to two reasons:

1. The remoting layer uses http communication, removing the need for custom encoding and decoding.
2. Spring’s HttpInvoker is leveraged to encapsulate the logic of refer and exporter.

What is this Spring HttpInvoker? Admittedly, it’s a rather obscure concept, but not complex. In simple terms, it uses Java serialization to convert an object into bytes, sends it over http, and on the server side, Spring can map the URL to find the corresponding Bean for reflective invocation. Don’t worry if you haven’t come across it before; you can quickly grasp this concept through the example below.

Spring HttpInvoker

This section references the Spring documentation: https://docs.spring.io/spring/docs/4.3.24.RELEASE/spring-framework-reference/htmlsingle/#remoting-httpinvoker-server

The following example will demonstrate how to achieve remote calls using Spring’s native HttpInvoker.

Creating a Service Provider

Code language: javascriptCopy

public class AccountServiceImpl implements AccountService {    @Override    public Account findById(int id) {       Account account = new Account(id, new Date().toString());       return account;    }}

Code language: javascriptCopy

@BeanAccountService accountService(){   return new AccountServiceImpl();}@Bean("/AccountService")public HttpInvokerServiceExporter accountServiceExporter(AccountService accountService){   HttpInvokerServiceExporter exporter = new HttpInvokerServiceExporter();   exporter.setService(accountService);   exporter.setServiceInterface(AccountService.class);   return exporter;}

The code to expose the service is relatively straightforward, with two key points:

1. org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter is a service exporter encapsulated by Spring, which provides a service externally with serviceInterface as the public interface and service as the implementing class.
2. @Bean(“/AccountService”) not only specifies the bean’s name in the IOC container but also serves a path mapping function. If the local server is exposed on port 8080, the service access path for the sample service is http://localhost:8080/AccountService

Creating a Service Consumer

Code language: javascriptCopy

@Configurationpublic class HttpProxyConfig {    @Bean("accountServiceProxy")    public HttpInvokerProxyFactoryBean accountServiceProxy(){       HttpInvokerProxyFactoryBean accountService = new HttpInvokerProxyFactoryBean();       accountService.setServiceInterface(AccountService.class);       accountService.setServiceUrl("http://localhost:8080/AccountService");       return accountService;    }}

Code language: javascriptCopy

@SpringBootApplicationpublic class HttpClientApp {    public static void main(String[] args) {       ConfigurableApplicationContext applicationContext = SpringApplication.run(HttpClientApp.class, args);       AccountService accountService = applicationContext.getBean(AccountService.class);       System.out.println(accountService.findById(10086));    }}

On the consumer side, referencing the service also has two noteworthy points:

1. org.springframework.remoting.httpinvoker.HttpInvokerProxyFactoryBean is a service referencer encapsulated by Spring, with serviceInterface specifying the interface for proxy generation and serviceUrl specifying the address where the service is located, corresponding with the previously configured service exporter’s path.
2. When HttpInvokerProxyFactoryBean is registered into the container, an AccountService interface proxy class is generated simultaneously with Spring encapsulating the logic of remote calls.

Analysis of Call Details

Delving deep into the underlying implementation of Spring HttpInvoker is unnecessary, but some might still be curious about certain details: How are the http message bodies in Dubbo organized? How are objects serialized?

Using Wireshark, we can capture the request sent by the client as well as the server’s response messages.

>

Tracking the message flow reveals detailed request and response content.

From ContentType:application/x-java-serialized-object and the ASCII code in the message body, we know Java Serialize is used for serialization. We export the body part into a file, use Java Serialize to deserialize the response, and verify what it truly looks like:

Using Java Serialize, we can normally deserialize the message, obtaining the result as Spring’s built-in wrapping class RemoteInvocationResult, which decorates the actual business return result.

Significance of the HTTP Protocol

Dubbo provides numerous protocols, each suited to specific scenarios. For example:

• dubbo://, the dubbo protocol, is the default protocol. It’s a custom binary protocol with a single long connection saving resources; based on TCP, built on Netty, with fairly good performance. Its protocol design doesn’t have sufficient foresight, not suitable for service-mesh, and isn’t particularly elegant, but it’s been in use for many years with many accompanying components like dubbo2.js, dubbo-go, and dubbo-cpp, which partially solve the multi-language problem.
• webservice://, hession://, thrift://, etc., these protocols mainly adapt existing server/client protocols. By leveraging Dubbo framework’s API, its functional features are usable, but the significance isn’t especially substantial.
• redis://, memcached://, etc., are neither protocols configured for user exposure. They are generally used internally within Dubbo, utilized in the registration center module with corresponding extensions.

The specific usage scenarios and features of all protocols might be analyzed in a separate article. For now, our concern is evaluating what problem Dubbo’s http protocol solves and under what circumstances users might consider using Dubbo’s http protocol.

In my personal opinion, the existing http protocol in Dubbo is relatively redundant. The native advantage of http communication lies in its universality, with nearly every language supporting http client and server. However, Dubbo’s http protocol utilizes application/x-java-serialized-object as the default payload format, causing it to lose its cross-language advantage. Some readers might argue: HttpInvoker supports serialization format customization, so it shouldn’t be criticized hastily. But actually, what we focus on is the default implementation, just like how the dubbo:// protocol can be configured to use fastjson as a serialization scheme, yet we similarly don’t consider the dubbo:// protocol as an excellent cross-language scheme, for the same reasons. Of course, evaluating whether an application layer protocol is excellent or suitable for a mesh, etc., needs analysis from multiple directions, which I’ll not cover in this article.

In the end, after spending considerable space introducing Dubbo’s http protocol, I aim to tell you this: It’s a relatively redundant protocol. Does this leave you somewhat disappointed? Don’t be disheartened. Dubbo might deprecate the existing http protocol in version 2.7.4 and instead adopt the jsonrpc protocol, which essentially is the jsonrpc protocol rebranded. I’ll discuss the details of jsonrpc in the next article, where I’ll also analyze why jsonrpc is more suited to take on the mantle of http protocol compared to the existing http protocol. As for the current http protocol, I am more inclined to call it: spring-httpinvoker protocol.

In summary, what is the significance of the existing Dubbo http protocol? Perhaps its existing usefulness lies for those accustomed to using Spring HttpInvoker, but given the small focus dedicated to it by Dubbo community discussions and Spring documentation, it remains quite niche. At the same time, it allows us to better understand the historical development of protocols, understanding why a protocol exists and why it gets phased out.

Of course, I’m not the final authority; it ultimately depends on the decision of the Dubbo community. If you’re interested in this migration plan and want to participate in the discussion, feel free to share your views on the Dubbo community’s mailing list.