Optional network interception capability in WebView

[Token-LiMing]

Optional network interception capability in WebView

Submitter(s)

Jiasheng Wu

Motivation

It‘s common to use WebView in Native App development. For example, some app will load one party websites for internal business, and will load third party website for other business. Apps want to intercept some internal request in WebView, to load offline resource, or just want to send request with native network framework. At the same time, third party request use WebView's networking process.
But not all WebView provide develop with optional network interception capability (WKWebView), which means the all request in WebView must use WebView's networking process.

Stakeholders

Stakeholders include:

WebView provider: Apple, Google, deciders on how to enhance WebView's networking intercept capability
WebView user: Native Business rely on WebView to send internal request, or load offline resource
End user: users of the App indirectly using the Features implemented by WebView.

Analysis

shouldInterceptRequest in Android WebView provide develop with optional network interception capability.

Related W3C deliverables and/or work items

Perhaps the specification could mandate that all WebViews must provide develop with optional network interception capability.

[rayankans]

I really like the idea of having extra control over first party websites. It would require some cross-platform way of associating an origin with the native/host applications.

[QingAn]

I really like the idea of having extra control over first party websites. It would require some cross-platform way of associating an origin with the native/host applications.

I see the origin of WebView has been mentioned many times. Maybe we can start with discussing the issues behind it?

[JohnRiv]

I believe my particular use case I'm interested in would fall under this Issue, but I can open a separate issue if the group feels it's different enough:

In a hybrid Native/Webview app, the native app and the webview may make the exact same HTTP calls to talk to an API. A way to share those HTTP requests/responses between Native & Webview would be useful. I think an ideal case is specifying specific URLs that can be shared (with flexibility similar to proxy software) and then when the webview or Native makes an HTTP request that matches one of those URL patterns, the response can be shared by Native and Webview if either makes a subsequent request that the HTTP caching rules would result in the cached version being returned.

[Token-LiMing]

I believe my particular use case I'm interested in would fall under this Issue, but I can open a separate issue if the group feels it's different enough:

In a hybrid Native/Webview app, the native app and the webview may make the exact same HTTP calls to talk to an API. A way to share those HTTP requests/responses between Native & Webview would be useful. I think an ideal case is specifying specific URLs that can be shared (with flexibility similar to proxy software) and then when the webview or Native makes an HTTP request that matches one of those URL patterns, the response can be shared by Native and Webview if either makes a subsequent request that the HTTP caching rules would result in the cached version being returned.

In limited business scenarios, We can try to share response data. For example, the client requests static resources from the backend. Request should be idempotent. It's possible to intercept those requests with java objective-c or swift code, send those request with business network library, perform optimization logic through the business network library.

中文对照：
我们可以在有限的业务场景下尝试分享响应数据。例如客户端向后端请求静态资源的场景。这个时候请求必须是幂等的，发送一次请求和两次请求对后端的数据库及业务逻辑不会产生语义上的变更。也许我们可以按照业务的要求将具有这些特征的请求代理到native这儿，通过充分业务网络库执行优化逻辑。

[QingAn]

I believe my particular use case I'm interested in would fall under this Issue, but I can open a separate issue if the group feels it's different enough:

I think the use case you mentioned is kind of different. It is more about the responses can be shared by Native and WebView, while the network interception is about that Native handles the resource request on behalf of WebView and the corresponding response data returned to WebView can even be different with what is received by Native from the backend server.

We can further discuss in next week teleconference

[JohnRiv]

Thanks for the reply. I won't be able to attend the upcoming teleconference and I'm in the US Eastern Time Zone. I'll open a separate issue for my topic so we can discuss in there and/or at a later teleconference at a different time.

[QingAn]

@Token-LiMing As discussed in CG meeting 2022-05-11, please add detailed use case on network interception by 3rd party.

[Token-LiMing]

@Token-LiMing As discussed in CG meeting 2022-05-11, please add detailed use case on network interception by 3rd party.

English description:
We expect to discuss the optional network interception capabilities in three perspectives.

1. Technical Perspective
   The core capability we need: WebView provides a client-side network proxy that can proxy requests to the client as needed and by client code, and the client returns the response to the WebView.
   After clarifying the core capability, we can carry out a layer of abstraction: the value of the proxy is the value of the optional network interception, and the capability scope of the proxy is the capability scope of the optional network interception.
   Compared with the horizontal technology stack, the operating system also provides proxy capabilities, which are used in a wide range of scenarios, security and reliability can be guaranteed. The provision of optional network interception capability helps the promotion of WebView.
   With the deep digging and abstraction of the technical perspective, you can precipitate use cases on this basis and continue to think backwards.

2. Business Perspective
   The concept of one-party and three-party is not a concept from a purely technical perspective, and must be accurately defined in combination with business scenarios. However, the development of the business must be based on technology, and based on the realization of the optional network interception capability, we can continue to discuss the final use case in the business.
   The value of optional network interception to one- and two-party services has been described above, and will not be repeated here. The following describes the value of optional network interception to third-party network requests from a business perspective in detail.
   Based on this ability, we can do sth. in the following scenarios:
   (1) Children's filter: If the user of the app is a child, by intercepting the request to the client, the client can connect with the security service to filter out the content that children are uncomfortable with, strengthen the security of web access.
   (2) Document access control: In the App, there is usually the concept of users. We expect to manage the access control of Web access documents, allowing different users to access documents of different levels.
   (3) Security firewall: In order to meet compliance requirements, network engineers want to audit requests within the app, and do not allow insecure requests to be sent on the web.
   (4) Web caching: For some third-party resources that are frequently accessed in the web, we hope to implement active caching to improve the speed of web access.
   (5) Transcoder: In special scenarios, we may need to convert GIF images to JPEG images to send to reduce traffic, and may also need to translate documents to meet communication needs in different language scenarios. For example, is's hard for me to communicating with English, when communicating, type Chinese, and send it out and hope that it will be automatically translated into English.
   (6) Anonymous: Some users may have higher requirements for the privacy of Web access, requiring that requests within the Web must remove identity features (such as client IP address, From header, Referer header, cookie, URI session ID), providing a high degree of privacy and anonymity.

3. User perspective
   Abstracting from technical capabilities and business demands, the final product is of great help to web end users, and can provide a web experience with higher compatibility, compliance, performance, privacy protection, and high availability.

Chinese description:
在线下和同事们进行了广泛的讨论，目前产生了一些新的意见。
针对可选的网络拦截能力，我们期望分为三个视角进行讨论。

1. 技术视角
   我们所需要的核心能力是：WebView 提供客户端网络代理，可以按照需要将请求代理到客户端并且由客户端代码进行，并且由客户端将响应返回到WebView。
   明确了核心能力后我们可以进行一层抽象：代理的价值就是可选网络拦截的价值，代理的能力范围就是可选网络拦截的能力范围。
   从横向技术栈进行对比，操作系统也提供了代理能力，在广大场景得到了运用，安全和可靠性是可以保障的。可选网络拦截能力的提供有助于WebView的推广。
   有了技术视角的深挖和抽象，就可以在这个基础上沉淀用例，继续向后思考。

2. 业务视角
   一二方和三方的概念并不是一个纯技术视角的概念，必须得结合业务场景才能做出准确的定义。但是业务的发展必须以技术为基石，基于可选网络拦截能力的实现才可以继续讨论业务上最终用例。
   上文已经描述过可选网络拦截对一二方业务的价值，此处不与赘述，下面详细的从业务视角描述可选网络拦截对三方网络请求的价值。
   基于此能力，我们可以在以下场景发力：
   (1) 儿童过滤器：如果 App 的受众是儿童，通过将请求拦截到客户端的方式，客户端和安全服务对接，可以过滤掉儿童不适的内容，进一步加强Web访问的安全性。
   (2) 文档访问控制：在 App 内，通常会有用户的概念，我们期望对 Web 访问文档进行访问控制管理，允许不同的用户访问不同等级的文档。
   (3) 安全防火墙：为了满足合规要求，网络工程师希望在 App 内的请求进行审计，不允许不安全的请求在 Web 中发送。
   (4) Web 缓存：针对部分在 Web 内高频访问的三方资源，我们希望进行主动缓存，以提升 Web 访问速度。
   (5) 转码器：特殊场景下，我们可能需要将 GIF 图片转换为 JPEG 图片发送以降低流量，可能也需要对文档进行翻译，以满足不同语言场景下的沟通需求。比如在下英文水平只通六窍，交流的时候打中文，发送出去希望自动翻译成英文。
   (6) 匿名者：部分用户可能对 Web 访问的私密性有较高的要求，要求 Web 内的请求得从 HTTP 报文中删除身份特征（比如客户端 IP 地址、From 首部、Referer 首部、cookie、URI 的会话 ID），从而提供高度的私密性和匿名性。

3. 用户视角
   从技术能力和业务诉求进行抽象，最终构建出的产品对于 Web 终端用户大有助益，可以提供兼容性更高、更合规、更高性能、更保护隐私、高可用的 Web 体验。

[QingAn]

(1) Children's filter: If the user of the app is a child, by intercepting the request to the client, the client can connect with the security service to filter out the content that children are uncomfortable with, strengthen the security of web access.
(2) Document access control: In the App, there is usually the concept of users. We expect to manage the access control of Web access documents, allowing different users to access documents of different levels.
(3) Security firewall: In order to meet compliance requirements, network engineers want to audit requests within the app, and do not allow insecure requests to be sent on the web.
(4) Web caching: For some third-party resources that are frequently accessed in the web, we hope to implement active caching to improve the speed of web access.
(5) Transcoder: In special scenarios, we may need to convert GIF images to JPEG images to send to reduce traffic, and may also need to translate documents to meet communication needs in different language scenarios. For example, is's hard for me to communicating with English, when communicating, type Chinese, and send it out and hope that it will be automatically translated into English.
(6) Anonymous: Some users may have higher requirements for the privacy of Web access, requiring that requests within the Web must remove identity features (such as client IP address, From header, Referer header, cookie, URI session ID), providing a high degree of privacy and anonymity.

I personally think these user scenarios make sense.

[QingAn]

Merge with #12 and close this issue.

[muodov]

@QingAn The interception/modifying/blocking capability is something DDG is interested in. I see that you closed this in favor or #12, but it sounds different to me (intercept and proxy vs. reuse responses with native). Should I post my perspective in #12, reopen this issue, or create a new one?

[QingAn]

@muodov This issue was closed at the request of the proposer. I think it is ok to create a new one.