Fine-tune the image encoding

By default, the service sends JPEG images at 70% quality for non-interactive mode, and at 30% quality for interactive mode (when enabled).

You can override the defaults. In addition to JPEG, the APIs support PNG, Tiles. and to a certain extent H.264. Each view in each client can have its own encoding configuration.

It is also possible to use other encoding formats, but this requires that you bypass the imaging pipeline.

If you would like to experiment first with various configurations, the Options tab in the Diagnostics Panel is a great tool to see the effect of your encoding changes without the need to edit the code or recompile between each try.

How-to

Enable the interactivity mode

APIs:

ViewManager.SetViewInteracting() | C++ | .Net | Java

The interactivity mode is a feature that allows you to use a different encoding configuration for the images when the user is interacting with a view. This is entirely optional.

This feature is generally used to send images of lower quality during interaction, since the images are being updated very rapidly (several per second) and the end user doesn't really see these intermediate renderings. This is a great way to improve performance without sacrificing the quality of the images that the user actually does see.

To implement this, you need to tell the service when user interaction begins and when it stops. This is achieved by setting the value for the SetViewInteracting flag to true (to turn interactive mode on) or false (to turn interactive mode off). For example, you could enable the interactive mode on a mousedown event, and turn it back off on a mouseup event.

C++

void MyViewClass::OnLButtonDown(UINT nFlags, CPoint point)
{
    m_startpt.x = point.x;
	m_startpt.y = point.y;
    // Tell the StateManager that the user is interacting with the view
    MyApp::StateManager().ViewManager().SetViewInteracting("Spectacular3D", true);
}
void MyViewClass::OnLButtonUp(UINT nFlags, CPoint point)
{
    m_startpt = -1;
	CView::OnLButtonUp(nFlags, point);
    // Tell the StateManager that the user finished interacting with the view
    MyApp::StateManager().ViewManager().SetViewInteracting("Spectacular3D", false);
}

.Net

protected override void OnMouseDown(MouseEventArgs e)
{
    if (e.Button == MouseButtons.Left || e.Button == MouseButtons.Right)
    {
        // Tell the StateManager that the user is interacting with the view
        Program.StateManager.ViewManager.SetViewInteracting("Spectacular3D", true);
        Capture = true;
        (...)
    }
}

protected override void OnMouseUp(MouseEventArgs e)
{
    if (e.Button == MouseButtons.Left || e.Button == MouseButtons.Right)
    {
        // Tell the StateManager that the user finished interacting with the view
        Program.StateManager.ViewManager.SetViewInteracting("Spectacular3D", false);
        Capture = false;
        (...)
    }
}

Java

public void mousePressed(MouseEvent e)
{
    isLeftButtonDown = e.getButton() == MouseEvent.BUTTON1;
	if (isLeftButtonDown)
	{
        // Tell the StateManager that the user is interacting with the view
        StateManager.getInstance().getViewManager().setViewInteracting("Spectacular3D", true);
        (...)
	}
}

public void mouseReleased(MouseEvent e)
{
    if (e.getButton() == MouseEvent.BUTTON1)
    {
        isLeftButtonDown = false;
        // Tell the StateManager that the user finished interacting with the view
        StateManager.getInstance().getViewManager().setViewInteracting("Spectacular3D", false);
        remoteRender();
    }
}
						

Once the interactive mode is set, you can specify a separate image encoding configuration for this mode, as explained in the next section.


Change the encoding configuration

APIs:

EncoderFormat | HTML5 | iOS | Android

EncoderConfiguration | HTML5 | iOS | Android

View.SetEncoderConfiguration() | HTML5 | Android
PWViewDelegate.preferredEncoderConfigurationForView() | iOS

The service provides an EncoderConfiguration object that determines the default image encoding for all views. On the client, you can override the service-provided default for a view by setting a custom EncoderConfiguration object. This approach makes it possible to support platform-specific preferred response type (JavaScript, for example, does not support tiles).

You could create your own EncoderConfiguration object on the service, although this is not common; this would set a new default configuration for all views in all clients.

Each EncoderConfiguration object contains two instances of EncoderFormat, one for the non-interactive (or full quality) mode, and one for the interactive mode. (Values for the interactive mode will be ignored if this mode has not been enabled on the service).

Each EncoderFormat, in turn, contains one or more arguments: the format (or mime type) is the only one which is mandatory. The APIs support JPEG (the default), PNG, Tiles and, to some extent, H.264. It is common to also specify a quality value (an integer between 0 and 100 that defines the overall image fidelity). And finally, you can also add custom parameters of your own, as described further down this page.

Setting a quality value for PNG images has no effect, since PNGs use loss-less encoding.

To set a view-specific encoding configuration:

  1. Create an EncoderFormat object for the full quality mode, and a second EncoderFormat object for the interactive mode. (The one for the interactive mode will be ignored if this mode has not been enabled on the service)

    HTML5

    var fullQuality = new pureweb.client.EncoderFormat('image/jpg', 100);
    var interactive = new pureweb.client.EncoderFormat('video/h264', 10);
    

    iOS (Obj-C)

    PWEncoderFormat *fullQuality = [PWEncoderFormat formatWithMimeType:@"image/jpg" quality:100];
    PWEncoderFormat *interactive = [PWEncoderFormat formatWithMimeType:@"image/tiles" quality:45];			

    Android

    EncoderFormat fullQuality = new EncoderFormat(SupportedEncoderMimeTypes.Jpeg, 100);
    EncoderFormat interactive = new EncoderFormat(SupportedEncoderMimeTypes.Tiles, 45);
    

  2. Create an EncoderConfiguration object using the two EncoderFormat you created in the previous step.

    HTML5

    var encoderConfig = new  pureweb.client.EncoderConfiguration(interactive, fullQuality);
    

    iOS (Obj-C)

    PWEncoderConfiguration *config = [PWEncoderConfiguration configurationWithInteractiveQuality:interactive fullQuality:fullQuality]
    

    Android

    EncoderConfiguration encoderConfig = new EncoderConfiguration(interactive, fullQuality);

  3. Assign the new EncoderConfiguration object to a specific view. This is done using View.SetEncoderConfiguration() in HTML5 and Android, or PWViewDelegate.preferredEncoderConfigurationForView() in iOS.

    HTML5

    Spectacular3D.setEncoderConfiguration(encoderConfig);
    

    iOS (Obj-C)

    Here is the final code, which includes the lines from the earlier steps:

    - (PWEncoderConfiguration *)preferredEncoderConfigurationForView:(PWView *)Spectacular3D
    {
        PWEncoderFormat *fullQuality = [PWEncoderFormat formatWithMimeType:@"image/jpeg" quality:100];
        PWEncoderFormat *interactiveQuality = [PWEncoderFormat formatWithMimeType:@"image/tiles" quality:45];
        PWEncoderConfiguration *config = [PWEncoderConfiguration configurationWithInteractiveQuality:interactiveQuality fullQuality:fullQuality];
        return config;
    }			

    Android

    Spectacular3D.setEncoderConfiguration(encoderConfig);


Add custom encoding parameters

When you specify an encoder format on the client, you can add your own custom encoding parameters. The service can then use this information to make rendering decisions.

For example, let's say that you would like the images in a given view to display a watermark. On the client, you could add a custom boolean parameter called "useWatermark". On the service, you would check the value for this parameter when renderView is called, and if it's true, you would execute some code to add a watermark to the images being rendered.

Write the parameters on the client

APIs:

EncoderFormat | HTML5 | Android
PWEncoderFormat | iOS

When you instantiate an EncoderFormat object, you can add custom parameters as the third, optional, argument. The parameters are expressed as an array of key-value strings. You can convert the parameters to different data types when you retrieve them from the service.

HTML5

var MyCustomEncoderFormat = new pureweb.client.EncoderFormat('image/jpg', 100, {'useWatermark':true});

iOS (Obj-C)

NSDictionary *params = @{@"useWatermark",@YES};
PWEncoderFormat *MyCustomEncoderFormat = [PWEncoderFormat formatWithMimeType:@"image/jpg" quality:100 parameters:params];

Android

Map<String, Object> parameters = new HashMap<String, Object>();
parameters.put("useWatermark", "true");
EncoderFormat MyCustomEncoderFormat = new EncoderFormat(SupportedEncoderMimeTypes.Jpeg, 100, parameters);

Under the hood, this adds the key-value pairs to the parameter map on RenderTarget.

Read the parameters from the service

APIs:

RenderTarget.Parameters | C++ | .Net
RenderTarget.getParameters | Java

To retrieve the custom encoding parameters on the service, simply call the Parameters method (called getParameters in Java) on the RenderTarget class.

C++

String useWatermark = target.Parameters()["useWatermark"];

.Net

String useWatermark = target.Parameters["useWatermark"];

Java

String useWatermark = target.getParameters()["useWatermark"];


Override the default image format

APIs:

ViewManager.setViewImageFormat() | C++ | .Net

Each framework provides default values for the characteristics of an image (pixel format, scan line order and alignment). The defaults vary based on the platform:

  C++ C# Java
PixelFormat Bgr24 Rgb24 Bgr24
ScanLineOrder TopDown TopDown TopDown
Alignment 4 4 4

When working with a C++ or C# service, if you need to override these defaults, you can do this using the SetViewImageFormat method.

ViewImageFormat viewImageFormat;
viewImageFormat.PixelFormat = PixelFormat::Rgb24;
viewImageFormat.ScanLineOrder = ScanLineOrder::TopDown;
viewImageFormat.Alignment = 4;
PureWebCommon::StateManager().ViewManager().SetViewImageFormat("MyAwesomeView", viewImageFormat);

Bypass the imaging pipeline

Some applications have very specific needs not met by the default imaging pipeline. For example if you want to:

  • use a different image format other than ones provided by the APIs,
  • use a custom encoder format,
  • pass arbitrary binary data (such as already compressed images, documents, and so on).

For these situations, it is possible to bypass the imaging pipeline. However, this requires that you provide a custom renderer implementation, on every client platform that you want to support, and is not a trivial undertaking.

Here are some pointers to get you started; please refer to the API library reference for your chosen platform for more information.

  • On the service, you will need to create a custom implementation of EncoderFormat. Also, when implementing RenderView, use either RenderTarget.ContentInfo or RenderTarget.Type (instead of the usual RenderTarget.Image).
  • On the client, you will need to create your own implementations of CustomRenderer, which is an extension of ViewRenderer. If you support more than one client (HTML5 and iOS for example), you must create a custom implementation on each client.