Android screenshots and WebView long pictures sharing experience summary

While working on new business needs recently, we encountered some problems on Android that we had never encountered before, such as screenshot sharing, long images generated by WebView, and blurry or even failed long images when shared on various sharing channels. We stepped on many pitfalls in the process, but so far, most of the problems have been solved satisfactorily. The following summarizes the challenges encountered in the process and the best solutions from three aspects.

1. Overview

While working on new business needs recently, we encountered some problems on Android that we had never encountered before, such as screenshot sharing, long images generated by WebView, and blurry or even failed long images when shared on various sharing channels. We stepped on many pitfalls in the process, but so far, most of the problems have been solved satisfactorily. The following summarizes the challenges encountered in the process and the best solutions from three aspects.

2. Screenshot sharing

There is no screenshot broadcast or monitoring event in the Android native system, which means that the user's screenshot operation cannot be known at the code level, so the user's need to pop up a sharing prompt after taking a screenshot cannot be met. Since the problem of screenshot monitoring cannot be fundamentally solved, we must consider implementing it indirectly through other methods. The more mature and stable solution is to monitor the changes in the system media database resources. The specific solution principle is as follows:

The Android system has a media database. Every time you take a photo or use the system screenshot function to capture an image, the detailed information of the image is added to the media database and a content change notification is issued. We can use content observers to monitor changes in the media database. When the database changes, we can obtain the newly inserted image data. If the image meets specific rules, it is considered to have been captured.

Considering that the mobile phone storage includes internal memory and external memory, in order to enhance compatibility, it is necessary to monitor the changes of both storage spaces at the same time. The following are the resource URIs that need to be monitored by ContentObserver:

 MediaStore.Images.Media.INTERNAL_CONTENT_URI  
 MediaStore.Images.Media.EXTERNAL_CONTENT_URI

To read external storage resources, you need to add permissions:

 android.permission.READ_EXTERNAL_STORAGE

Note: On Android 6.0 and above, you need to apply for permissions dynamically.

1. Screenshot judgment rules

When ContentObserver monitors data changes in the media database, it obtains the image data that was first inserted into the database when there is a data change. If it meets the following rules, it is considered a screenshot:

1. Time judgment: Usually, the screenshot will be stored in the system multimedia database immediately after it is generated. That is to say, the time when the database changes are monitored will not differ too much from the time when the screenshot is generated. Here, 10 seconds is recommended as the threshold. Of course, this is also an empirical value.
2. Size judgment: As the name suggests, a screenshot is a picture of the current size of the mobile phone screen, so pictures whose width and height are greater than the width and height of the screen are definitely not screenshots.
3. Path judgment: Since the file paths for storing screenshots are different for each mobile phone manufacturer, the situation in China may be more serious. However, usually the image saving path will contain some common keywords, such as "screenshot", "screencapture", "screencap", "screenshot", "screenshot", etc. Check the image path information every time to see if it contains these keywords.

Regarding point 3, I need to explain that since it is necessary to determine whether the image file path contains keywords, it currently only supports Chinese and English environments. If you need to support other languages, you need to manually add some keywords of the language, otherwise you may not be able to obtain the image.

The above three points can basically ensure the normal monitoring of screenshots. Of course, in the actual testing process, it will be found that some models have multiple reports, so some deduplication work is still needed. Deduplication will be mentioned again below.

2. Key code

Now that the principles are clear, the next question is how to implement it. The key here is to set up the media content observer, take out the first piece of data from the database and parse the image information, and then check whether the image information meets the above three rules.

In order to explain how to monitor changes in the media database, we first need to briefly talk about the principle of ContentObserver. ContentObserver - content observer, the purpose is to observe (capture) changes in the database caused by a specific Uri, and then do some corresponding processing. It is similar to the trigger in database technology. When the Uri observed by ContentObserver changes, it will be triggered. Of course, if you want to observe, you must first register. The Android system provides the ContentResolver#registerContentObserver method to register the observer. Students who are not familiar with this part can review the knowledge related to Android's ContentProvider.

Next, we will use code to illustrate the entire registration and triggering process. The code is as follows:

 private void initMediaContentObserver() {
 // Handler running on the UI thread, used to run the listener callback
    private final Handler mUiHandler = new Handler(Looper.getMainLooper());
 // Create content observers, including internal storage and external storage
    mInternalObserver = new MediaContentObserver(MediaStore.Images.Media.INTERNAL_CONTENT_URI, mUiHandler);
    mExternalObserver = new MediaContentObserver(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, mUiHandler);
 // Register content observer
    mContext.getContentResolver().registerContentObserver(
            MediaStore.Images.Media.INTERNAL_CONTENT_URI, false , mInternalObserver);
    mContext.getContentResolver().registerContentObserver(
            MediaStore.Images.Media.EXTERNAL_CONTENT_URI, false , mExternalObserver);
 }
 /**
 * Custom media content observer class (observe changes in the media database)
 */
 private class MediaContentObserver extends ContentObserver {
 private Uri mediaContentUri; // Uri to be observed
 public MediaContentObserver(Uri contentUri, Handler handler) {
 super(handler);
        mediaContentUri = contentUri;
 }
 @Override
 public void onChange(boolean selfChange) {
        super.onChange(selfChange);
 // Process data changes reported by the media database
        handleMediaContentChange(mediaContentUri);
 }
 }

If there is a registration, it needs to be unregistered when the Activity is destroyed, so it is also necessary to encapsulate a deregistration method for external calls. The Android system provides the ContentResolver#unregisterContentObserver method to cancel the registration. The code is relatively simple and will not be shown here.

After the listener is set up and registered, once the user takes a screenshot, the system will execute the ContentObserver#onChange callback method, in which we can obtain and parse data based on the Uri. Here we will show the specific data parsing process. The above-mentioned rule judgment is relatively simple and will not be shown here.

 private void handleMediaContentChange(Uri contentUri) {
 Cursor   cursor = null ;
 try {
 // When the data changes, query the database for the last piece of data added
 cursor = mContext.getContentResolver().query(contentUri,
                    Build.VERSION.SDK_INT < 16 ? MEDIA_PROJECTIONS : MEDIA_PROJECTIONS_API_16,
 null , null , MediaStore.Images.ImageColumns.DATE_ADDED + " desc limit 1" );
            if ( cursor == null ) return ;
            if (! cursor .moveToFirst()) return ;
 // cursor .getColumnIndex gets the database column index
 int dataIndex = cursor .getColumnIndex(MediaStore.Images.ImageColumns.DATA);
 String data = cursor .getString(dataIndex); // Image storage address 
 
 int dateTakenIndex = cursor .getColumnIndex(MediaStore.Images.ImageColumns.DATE_TAKEN);
 long dateTaken = cursor .getLong(dateTakenIndex); // Image generation time 
 
 int width = 0;
 int height = 0;
            if (Build.VERSION.SDK_INT >= 16) {
 int widthIndex = cursor .getColumnIndex(MediaStore.Images.ImageColumns.WIDTH);
 int heightIndex = cursor .getColumnIndex(MediaStore.Images.ImageColumns.HEIGHT);
 width = cursor .getInt(widthIndex); // Get the image height
 height = cursor .getInt(heightIndex); // Get the image width
 } else {
 Point size = getImageSize(data); // Get the image width and height based on the path
                width = size .x;
                height = size .y;
 }
 // Process the first line of data obtained, and determine whether the path contains keywords, time difference, and the size relationship between the image width and height and the screen width and height
            handleMediaRowData(data, dateTaken, width, height);
        } catch (Exception e) {
            e.printStackTrace();
 finally
            if ( cursor != null && ! cursor .isClosed()) {
 cursor.close ( ) ;
 }
 }
 }

Some mobile phone ROMs will send multiple content change notifications when taking a screenshot, so deduplication is required. Deduplication is not complicated. You can use a list to cache the latest dozen or so image address data. Every time a new image address is obtained, it will first determine whether the same image address exists in the cache. If the current image address already exists in the list, it can be directly filtered out, otherwise it will be added to the cache. In this way, it can be ensured that the screenshot monitoring event is neither missed nor repeated.

The above is the core principle and key code of mobile phone screenshot. If you need to share the screenshot, it is also very simple. Data is the storage address of the picture, which can be converted into Bitmap to complete the sharing.

2. WebView generates long images

Before introducing web long images, let's first talk about the single-screen image generation solution. Unlike mobile phone screenshots, the generated images will not display the top status bar, title bar, and bottom menu bar, which can meet different business needs.

 // WebView generates an image of the current screen size, shortImage is the final generated image
 Bitmap shortImage = Bitmap.createBitmap(screenWidth, screenHeight, Bitmap.Config.RGB_565);
 Canvas canvas = new Canvas(shortImage); // The width and height of the canvas are consistent with the width and height of the screen
 Paint paint = new Paint();
 canvas.drawBitmap(shortImage, screenWidth, screenHeight, paint);
 mWebView.draw(canvas);

Sometimes we need to generate a long web page into a picture and share it. A similar example is various note-taking applications on mobile phones. When the content of the note exceeds one screen, all the content needs to be generated into a long picture and shared.

WebView is the same as other Views. The system provides a draw method that can directly render the content of the View to the canvas. With the canvas, we can draw various other contents on it, such as adding a logo image at the bottom, drawing a red line frame, etc. There are many ready-made solutions and codes on the Internet for WebView to generate long images. The following code is a tested stable version for reference.

 // WebView generates long images, that is, images that exceed one screen. The longImage in the code is the long image generated by ***
 mWebView.measure(MeasureSpec.makeMeasureSpec(MeasureSpec.UNSPECIFIED, MeasureSpec.UNSPECIFIED),
                 MeasureSpec.makeMeasureSpec(0, MeasureSpec.UNSPECIFIED));
 mWebView.layout(0, 0, mWebView.getMeasuredWidth(), mWebView.getMeasuredHeight());
 mWebView.setDrawingCacheEnabled( true );
 mWebView.buildDrawingCache();
 Bitmap longImage = Bitmap.createBitmap(mWebView.getMeasuredWidth(),
        mWebView.getMeasuredHeight(), Bitmap.Config.ARGB_8888);
 Canvas canvas = new Canvas(longImage); // The width and height of the canvas should be consistent with the WebView webpage
 Paint paint = new Paint();
 canvas.drawBitmap(longImage, 0, mWebView.getMeasuredHeight(), paint);
 mWebView.draw(canvas);

In order to improve the drawing speed of scrolling and other aspects, Android can build a cache for each View. Use View#buildDrawingCache to build a corresponding cache for your View. This cache is a bitmap object. This function can be used to capture the entire screen view and generate a Bitmap, or to obtain the Bitmap object of a specified View. Here, since the Logo needs to be drawn on the original image, the draw method of WebView is used directly.

Since most of our H5 pages run in WeChat's X5 browser, in order to reduce the front-end adaptation work, we introduced Tencent's X5 browser kernel into the Android project to replace the system's native WebView kernel. There will be a special article introducing the introduction of the X5 kernel later, so stay tuned.

Here we need to explain how to generate long web images under the X5 kernel. The above code shows the solution of the system native WebView to generate images, but the above code is invalid in the X5 environment. After stepping on the pit and checking the X5 kernel source code, we finally found a solution to the problem. The following key code is used to illustrate the specific implementation method.

 // The mWebView here is the WebView of the X5 kernel, and the longImage in the code is the long image generated by ***
 mWebView.measure(MeasureSpec.makeMeasureSpec(MeasureSpec.UNSPECIFIED, MeasureSpec.UNSPECIFIED),
                 MeasureSpec.makeMeasureSpec(0, MeasureSpec.UNSPECIFIED));
 mWebView.layout(0, 0, mWebView.getMeasuredWidth(), mWebView.getMeasuredHeight());
 mWebView.setDrawingCacheEnabled( true );
 mWebView.buildDrawingCache();
 Bitmap longImage = Bitmap.createBitmap(mWebView.getMeasuredWidth(),
        mWebView.getMeasuredHeight() + endHeight, Bitmap.Config.ARGB_8888);
 Canvas canvas = new Canvas(longImage); // The width and height of the canvas should be consistent with the WebView webpage
 Paint paint = new Paint();
 canvas.drawBitmap(longImage, 0, mWebView.getMeasuredHeight(), paint);
 float scale = getResources().getDisplayMetrics().density;
 x5Bitmap = Bitmap.createBitmap(mWebView.getWidth(), mWebView.getHeight(), Bitmap.Config.ARGB_8888);
 Canvas x5Canvas = new Canvas(x5Bitmap);
 x5Canvas.drawColor(ContextCompat.getColor(this, R.color.fragment_default_background));
 mWebView.getX5WebViewExtension().snapshotWholePage(x5Canvas, false , false ); // Without this line of code, the long image cannot be generated normally
 Matrix matrix = new Matrix();
 matrix.setScale(scale, scale);
 longCanvas.drawBitmap(x5Bitmap, matrix, paint);

Note: The clarity of long images generated by the X5 kernel is slightly worse than that of native WebView, and there is currently no good solution.

3. Share long pictures

Generally, the pictures we send to various social platforms are relatively small, that is, the size of the mobile phone screen, and larger pictures are rare. However, there are exceptions, such as long pictures on Weibo and Hammer Notes. If you share these pictures directly through WeChat Sharing SDK or Weibo Sharing SDK, you will find that the pictures are basically blurry, but if you send the pictures to an iPhone, you can view them normally. We can only lament that the Android version of WeChat is not powerful enough.

WeChat SDK is not powerful, but the product experience cannot be lost. What should we do? There are ways. We all know that in addition to the sharing SDKs of various social platforms, the system provides a native sharing solution. In essence, the social platform exposes the target Activity to the outside world, and then the third-party App can call up the social platform according to the pre-defined Intent jump rules, and complete data transmission and display at the same time.

It seems that the problem can be solved completely, but there are still pitfalls to be stepped on. In Android 7.0 and above, the system restricts the transmission of data starting with file:// in Intent, which also restricts the system's native sharing of single pictures. What can we do? There are two solutions. One is to use WeChat and other sharing SDKs on 7.0 and above, and accept the current situation of blurry shared pictures. The other is to skip the system's restrictions on the transmission of files starting with file:// in Intent through reflection, but this method is risky. After all, we don't know what adjustments Android will make in the future. The following is a code snippet for skipping the system restrictions for reference.

 if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
 try {
        Method ddfu = StrictMode.class.getDeclaredMethod( "disableDeathOnFileUriExposure" );
 ddfu.invoke( null );
 } catch (Exception e) {
 }
 }

So far, it can basically meet the needs of sharing pictures of any size. In addition, it has been verified that WeChat sharing SDK for Android has restrictions on the size of thumbnails and shared pictures. The official guidance is that thumbnails smaller than 32K and shared pictures smaller than 10M can be shared normally, but after testing, these two values ​​are theoretical upper limits. Don't get too close to this upper limit. If the picture is too large, the thumbnail and the shared picture will appear blurry, or even unable to be shared normally. Of course, there is no such restriction for sharing through the system, and the picture is clearer.

In addition to the limit on image size, there is also a limit on the size of thumbnails. This is not given in the official documentation. Test results show that image sizes less than or equal to 120×120 are a relatively safe range and there is no problem sharing.

IV. Summary

Screen capture monitoring, WebView generation of long images, and long image sharing are all business requirements that our team has never encountered before. While meeting the product business requirements, we have also stepped on many pitfalls and accumulated some experience, which we would like to summarize here.