Optimization of iOS photo album Moment function

Recently, I am developing the Gallery module of my company's product Perfect365, including the Moment and Album modules sorted by date. The Moment function is similar to the system album, which is to sort the pictures according to their date information, and then display them in sections according to different dates.

The implementation idea of ​​Moment is very simple: first traverse all the albums in the system, then obtain the date information of the pictures in each album, classify and sort them according to the date, and finally put all the enumerated data on the interface for display. The sample code is as follows:

NSSortDescriptor *sort = [NSSortDescriptor sortDescriptorWithKey:@ "date" ascending:NO]; 
 
 [objects sortUsingDescriptors:@[sort]]; 
 
 MomentCollection *lastGroup = nil; NSMutableArray *ds = [[NSMutableArray alloc] init]; 
 
 for (ALAsset *asset in objects)
 {
 @autoreleasepool  
 {
        NSDateComponents *components = [[NSCalendar currentCalendar] components:NSDayCalendarUnit |
                                                                                NSMonthCalendarUnit |
                                                                                NSYearCalendarUnit
                                                                       fromDate:[asset date]];
        NSUInteger month = [components month];
        NSUInteger year = [components year];
        NSUInteger day = [components day]; 
         
 if (!lastGroup || lastGroup.year!=year || lastGroup.month!=month || lastGroup.day!=day)
 {
            lastGroup = [MomentCollection new ]; [ds addObject:lastGroup]; 
             
            lastGroup.month = month; lastGroup.year = year; lastGroup.day = day;
 } 
         
        ALAsset *lPhoto = [lastGroup.assetObjs lastObject];
        NSURL *lPhotoURL = [lPhoto valueForProperty:ALAssetPropertyAssetURL];
        NSURL *photoURL = [asset valueForProperty:ALAssetPropertyAssetURL];
 if (![lPhotoURL isEqual:photoURL])
 {
            [lastGroup.assetObjs addObject:asset];
 }
 }
 }

So far so good, next create UICollectionView, set dataSource and you can display moment pictures. I thought so at first, but everything is possible for developing an App with 65 million users. After the version was released, many users reported that the App freezes directly after opening the album, what the hell is this? Everything was OK during QA testing. Well, continue to harass users to ask what is going on, users reply: I have 30k+ pictures in my phone, which takes up 20G+ storage space. OH MY GOD!!!

Optimization plan

For the Moment function, it is necessary to traverse all the album pictures in the system, and then sort them by date and display them to the user. Then the optimization can only be squeezed in the two parts of enumeration and sorting. After two days of hard thinking, it was decided to use the solution of batch loading + tail sorting to optimize. The specific idea is: if there are many pictures in the user's device, instead of waiting for all pictures to be enumerated and sorted before displaying, we enumerate every certain number of pictures (eg 50 pictures) and throw them out (put them in NSOperationQueue) to classify and sort them by date, and then display them to the user, so that the user can see the process of our dynamic loading of pictures, let him know that our program is still alive and is constantly loading pictures. However, in general, the time taken for sorting will be longer than the enumeration of pictures, that is, after the first 50 pictures are sorted, there are already several batches of pictures enumerated and put in the Queue waiting for sorting, then we only sort the last batch of pictures (that is, tail) and clear the current Queue, because the middle batches of data already make no sense. The detailed flow chart of the solution is as follows:

Curve Flowchart

In order to minimize the abruptness of the refresh display after dynamic loading, it is necessary to determine whether the user is sliding the page before displaying, and refresh the display only when the page is still. However, the last batch of data after all pictures are enumerated must be temporarily saved (otherwise nothing will be displayed), and reloadData after the user stops sliding.

Batch loading

Moment needs to be displayed by date (the oldest is displayed at the front), so when enumerating the albums, you can start from the camera roll (the photos taken by users are usually earlier than the imported photos). After loading to a multiple of 50, it will be thrown into the queue to wait for sorting. After one album is enumerated, continue to traverse the rest of the albums...

- ( void )getPhotosWithGroupTypes:(ALAssetsGroupType)types
                    batchReturn:(BOOL)batch
                     completion:( void (^)(BOOL ret, id obj))completion
 {
    self.batchBlock = completion;
    NSMutableArray *tmpArr = [[NSMutableArray alloc] init]; 
     
    [self.assetLibary enumerateGroupsWithTypes:types 
     
                                    usingBlock:^(ALAssetsGroup *group, BOOL *stop)
 {
 if (self.stopEnumeratePhoto) {*stop = YES; return ;}
        NSInteger gType = [[group valueForProperty:ALAssetsGroupPropertyType] integerValue];
 if (group && (gType != ALAssetsGroupPhotoStream))
 {
            [group setAssetsFilter:[ALAssetsFilter allPhotos]]; 
             
            [group enumerateAssetsWithOptions:NSEnumerationReverse
                                   usingBlock:^(ALAsset *result, NSUInteger index, BOOL *stop)
 {
 if (self.stopEnumeratePhoto) {*stop = YES; return ;} 
                 
 if (result) [tmpArr addObject:result]; 
                 
 if (batch && !([tmpArr count]% 50 )) [self addQueueWithData:tmpArr final :NO];
 }];
 }
 else   if (nil == group)
 {
            [self addQueueWithData:tmpArr final :YES];
 }
 }failureBlock:nil];
 }

Tail sort

Each batch of images is added to a serial queue to wait for sorting. After a batch is sorted, the last one in the current queue (that is, the enumerated image that came last) is taken to continue sorting, and the current queue is cleared. That is, cleanQueueAfterRoundOperation is called in the sortMomentWithDate:final: function below.

- ( void )addQueueWithData:(NSMutableArray *)data final :(BOOL) final  
 {
    NSMutableArray *rawData = [NSMutableArray arrayWithArray:data]; 
     
    NSBlockOperation *op = [NSBlockOperation blockOperationWithBlock:^
 {
        [self sortMomentWithDate:rawData final : final ];
 }]; 
     
    [self.operQueue addOperation:op];
 } 
 
 - ( void )cleanQueueAfterRoundOperation
 {
 if (self.operQueue == nil) return ; 
     
 if (self.operQueue.operationCount > 1 )
 {
        NSArray *queueArr = self.operQueue.operations;
        NSMutableArray *opArr = [NSMutableArray arrayWithArray:queueArr]; 
         
        [opArr removeLastObject]; [opArr removeLastObject];
        [opArr makeObjectsPerformSelector: @selector (cancel)];
 }
 }

Refresh CollectionView to display pictures

After the middle batch of data classified by date is ready, before reloadData, first determine whether the current user is scrolling the collectionView. If it is not in the scrolling state, refresh the display, otherwise drop it directly. However, the last batch of data needs to be stored first and judged in scrollViewDidEndDragging and scrollViewDidEndDecelerating. Once the user stops sliding, it will be refreshed to the collectionView immediately.

[[ImageDataAPI sharedInstance] getMomentsWithBatchReturn:YES
                                               ascending:NO
                                              completion:^(BOOL done, id obj)
 {
    NSMutableArray *dArr = (NSMutableArray *)obj; 
     
 if (dArr != nil && [dArr count])
 {
 if (!self.momentView.dragging && !self.momentView.decelerating)
 {
            dispatch_async(dispatch_get_main_queue(), ^
 {
                [self reloadWithData:dArr];
 });
 }
 else  
 {
 if (done) {self.backupArr = dArr}
 }
 }
 }]; 
 
 - ( void )scrollViewDidEndDragging:(UIScrollView *)scrollView willDecelerate:(BOOL)decelerate
 {
 if (!decelerate && self.backupArr)
 {
        dispatch_async(dispatch_get_main_queue(), ^{
            [self reloadWithData:self.backupArr];
            self.backupArr = nil; // done refresh  
 });
 }
 } 
 
 - ( void )scrollViewDidEndDecelerating:(UIScrollView *)scrollView
 {
 if (self.backupArr)
 {
        dispatch_async(dispatch_get_main_queue(), ^{
            [self reloadWithData:self.backupArr];
            self.backupArr = nil; // done refresh  
 });
 }
 }

Subsequent improvement ideas

According to the above solution, no matter how many pictures there are on the device, the album can be opened and loaded normally. But there are still many areas that need to be improved:

• The data in the intermediate batch can also be stored after it is ready, and then reloaded after the user stops sliding, instead of simply dropping it.

• The sorting needs to be optimized. Now after the first batch of 50 pictures are sorted, the second batch of 200 pictures need to be reclassified and sorted. The middle batch data is just for displaying to users first. Can the 200th picture only sort the next 150 pictures, that is, if the next 150 pictures have new dates, create a new section, and directly insert the same date to the front. This needs to be studied later...

The above are just some of my own optimization ideas. If you have a better solution, please leave a message~~

Photos.framework

iOS 8 introduces a new PhotoKit API to replace the AssetsLibrary framework. PhotoKit provides an interface for directly accessing Moment data + (PHFetchResult *)fetchMomentsWithOptions:(nullable PHFetchOptions *)options This function directly returns a collection of pictures classified by date, and the speed is very fast (I guess Apple has marked the date information and sorted it by category when the user takes or imports the picture). Therefore, the PhotoKit framework can be directly used to implement the moment function in iOS 8 and above.

PHFetchOptions *options = [[PHFetchOptions alloc] init];
 options.sortDescriptors = @[[NSSortDescriptor sortDescriptorWithKey:@ "endDate"  
                                                           ascending:ascending]];
 PHFetchResult *momentRes = [PHAssetCollection fetchMomentsWithOptions:options];
 NSMutableArray *momArray = [[NSMutableArray alloc] init];
 for (PHAssetCollection *collection in momentRes)
 {
    NSDateComponents *components = [[NSCalendar currentCalendar] components:NSDayCalendarUnit |
                                                                            NSMonthCalendarUnit |
                                                                            NSYearCalendarUnit
                                                                   fromDate:collection.endDate];
    NSUInteger month = [components month];
    NSUInteger year = [components year];
    NSUInteger day = [components day];
    MomentCollection *moment = [MomentCollection new ];
    moment.month = month; moment.year = year; moment.day = day;
    PHFetchOptions *option = [[PHFetchOptions alloc] init];
    option.predicate = [NSPredicate predicateWithFormat:@ "mediaType = %d" , PHAssetMediaTypeImage];
    moment.assetObjs = [PHAsset fetchAssetsInAssetCollection:collection
                                                     options:option];
 if ([moment.assetObjs count]) [momArray addObject:moment];
 }

So, we can unify the moment interface externally and differentiate the system implementations internally (Gallery Model class): iOS 7 system uses AssetsLibrary and uses the above optimization solution, while iOS 8 system directly calls the Moment interface of Photos.framework.

But there is a problem here. The type of AssetsLibrary is ALAssetsGroup, and the type of PhotoKit is PHFetchResult. How can we unify them when using them? Do we need to distinguish the systems when calling externally?

The solution is very simple. Define your own data class, distinguish it within the data structure, and use the data type you defined when making external calls:

For example, define MomentCollection, including year, month, and day information, and the external attribute assetObjs distinguishes the system internally and returns or sets the corresponding type:

@interface MomentCollection : NSObject 
 
 @property (nonatomic, readwrite) NSUInteger month;
 @property (nonatomic, readwrite) NSUInteger year;
 @property (nonatomic, readwrite) NSUInteger day;
 @property (nonatomic, strong) id assetObjs; 
 
 @end   
 
 @property (nonatomic, strong) NSMutableArray *items;
 @property (nonatomic, strong) PHFetchResult *assets; 
 
 - (id)assetObjs
 {
 return IS_IOS_8 ? self.assets : self.items;
 } 
 
 - ( void )setAssetObjs:(id)assetObjs
 {
 if (IS_IOS_8)
 {
        self.assets = (PHFetchResult *)assetObjs;
 }
 else  
 {
        self.items = (NSMutableArray *)assetObjs;
 }
 }

The same processing method is used for albums or specific pictures. AlbumObj and PhotoObj data types are defined. In this way, the outside world (caller) does not need to worry about the data type, and all the logic is handled internally...

In addition, for other functions, such as enumerating albums, obtaining album poster images, obtaining image URLs, obtaining all thumbnails in an album, etc., a unified interface can be used externally, and internally distinguish whether to use PhotoKit or AssetsLibrary.

- ( void )getMomentsWithBatchReturn:(BOOL)batch // batch for iOS 7 only  
                        ascending:(BOOL)ascending
                       completion:( void (^)(BOOL done, id obj))completion; 
                        
 - ( void )getThumbnailForAssetObj:(id)asset
                       withSize:(CGSize)size // size for iOS 8 only  
                     completion:( void (^)(BOOL ret, UIImage *image))completion; 
                      
 - ( void )getURLForAssetObj:(id)asset
 /*usingPH:(BOOL)usingPH*/  
               completion:( void (^)(BOOL ret, NSURL *URL))completion; 
                
 - ( void )getAlbumsWithCompletion:( void (^)(BOOL ret, id obj))completion; 
 
 - ( void )getPosterImageForAlbumObj:(id)album
                       completion:( void (^)(BOOL ret, id obj))completion; 
                        
 - ( void )getPhotosWithGroup:(id)group
                completion:( void (^)(BOOL ret, id obj))completion; 
                 
 - ( void )getImageForPhotoObj:(id)asset
                   withSize:(CGSize)size
                 completion:( void (^)(BOOL ret, UIImage *image))completion;

The complete moment optimization solution and PhotoKit/AssetsLibrary integration interface implementation code (RJPhotoGallery) have been uploaded to GitHub. Interested friends can refer to it. The ImageDataAPI encapsulated in the program is the model class for image loading, which implements the Moment/Album function. If you need it, you can directly copy it and use it.

PS Welcome all the friends to comment and communicate~~