 |
|
 |
| |
 |
Single logical screen system and method for rendering graphical data |
| 6864894 |
Single logical screen system and method for rendering graphical data
|
|
| Patent Drawings: | |
| Inventor: |
Lefebvre, et al. |
| Date Issued: |
March 8, 2005 |
| Application: |
09/715,746 |
| Filed: |
November 17, 2000 |
| Inventors: |
Gee; Joseph Norman (Ft Collins, CO)
Hoffman; Don B. (Fort Collins, CO)
Lefebvre; Kevin (Ft Collins, CO)
Walls; Jeffrey J (Fort Collins, CO)
|
| Assignee: |
Hewlett-Packard Development Company, L.P. (Houston, TX) |
| Primary Examiner: |
Tung; Kee M. |
| Assistant Examiner: |
|
| Attorney Or Agent: |
|
| U.S. Class: |
345/1.1; 345/506 |
| Field Of Search: |
345/501; 345/502; 345/503; 345/504; 345/505; 345/506; 345/519; 345/520; 345/522; 345/530; 345/531; 345/532; 345/533; 345/534; 345/535; 345/536; 345/537; 345/538; 345/539; 345/540; 345/541; 345/542; 345/543; 345/544; 345/545; 345/546; 345/547; 345/548; 345/549; 345/550; 345/551; 345/552; 345/553; 345/554; 345/555; 345/556; 345/557; 345/558; 345/559; 345/560; 345/561; 345/562; 345/563; 345/564; 345/565; 345/566; 345/567; 345/568; 345/569; 345/570; 345/571; 345/572; 345/573; 345/574; 345/1.1; 345/1.3; 345/629; 345/419; 345/418 |
| International Class: |
G06T 1/20 |
| U.S Patent Documents: |
5408602; 5754242; 5757321; 5844553; 5956046; 6075917; 6084553; 6088036; 6111582; 6118433; 6188385; 6222550; 6252600; 6496186; 6501480; 6518971; 6573905; 6611241 |
| Foreign Patent Documents: |
|
| Other References: |
Steven Molnar et al "PixelFlow: High speed rendering using image composition", ACM Computer Graphics, 26, Jul. 2, 1992, pp. 231-240.*.
Computer Wall II, RGB Spectrum, Specifications, Sep. 2000, Alameda, CA 94501, http://www.rgb.com/Webpages/prodpgs/cwall.html.*.
Dachille, et al, GI-Cube: An Architecture for Volumetric Global Illumination and Rendering, Siggraph/Eurographics Workshop on Graphics Hardware, Aug. 2000, Interlaken Switzerland, ACM Press, NY, NY, pp. 119-128.*.
"Understanding X Features: Distributed Single Logical Screen" http://www.hp.com/xwindow/shardInfo/Whitepapers/Slsd/slsd.html, 1998, pp. 1-10..
Lefebvre, Kevin "Hewlett-Packard's Large Screen Multi-Display Technology: An Exploration of the Architecture Behind HP's New Immersive Visualization Solutions" http://www.hp.com;xwindow/sharedInfo/Whitepapers/Sls3d/sls_3d.html; 1998, pp. 1-9..
"Understanding X Features: Multiple Display Technologies" http://www.hp.com/xwindow/sharedInfo/Whitepapers/Sls/sls.html, 1997, pp. 1-13..
Alcorn et al., "Systems for Compositing Graphical Data," U.S. Appl. No. 09/715,892, filed Nov. 17, 2000..
Robertus et al., "Systems and Methods for Compositing Graphical Data," U.S. Appl. No. 09/715,232, filed Nov. 17, 2000..
Alcorn et al., "Systems and Methods for Rendering Graphical Data," U.S. Appl. No. 09/715,882, filed Nov. 17, 2000..
Alcorn et al., "Systems and Methods for Rendering Active Stereo Graphical Data as Passive Stereo," U.S. Appl. No. 09/715,600, filed Nov. 17, 2000..
Lefebvre et al., "System and Method for Efficiently Rendering Graphical Data," U.S. Appl. No. 09/715,335, filed Nov. 17, 2000..
Lefebvre et al., "System and Method for Efficiently Rendering a Jitter Enhanced Graphical Image," U.S. Appl. No. 09/715,253, filed Nov. 17, 2000.. |
|
| Abstract: |
A graphical display system utilizes a plurality of display devices and a plurality of graphical acceleration units for rendering graphical data to the display devices. More specifically, each of the plurality of graphical acceleration units respectively interfaces a portion of graphical data defining an image to one of the display devices. Each of the display devices displays a portion of the image based on the graphical data rendered to it. To make the system more efficient and/or to improve image quality, at least one of the graphical acceleration units includes a plurality of graphical pipelines for rendering the graphical data to be displayed by the display device that is interfaced with the one graphical acceleration unit. |
| Claim: |
What is claimed is:
1. A single logical screen (SLS) graphical system, comprising: an interface configured to receive graphical data defining an image; a plurality of display devices; and aplurality of graphical acceleration units, each of said plurality of graphical acceleration units respectively interfaced with one of said plurality of display devices and configured to render a portion of said graphical data to said one of saidplurality of display devices such that said display devices display said image as a single logical screen, wherein at least one of said graphical acceleration units comprises: a first graphical pipeline configured to receive and process a graphicalcommand, said first graphical pipeline configured to render and super-sample graphical data from said graphical command; a second graphical pipeline configured to receive and process said graphical command, said second graphical pipeline configured torender and super-sample graphical data; and a compositor interfaced with said first and second graphical pipelines and one of said display devices.
2. The system of claim 1, wherein said second graphical pipeline is configured to discard said graphical data rendered by said first graphical pipeline.
3. The system of claim 2, wherein said first graphical pipeline is configured to receive an input identifying a first coordinate range and is configured to discard graphical data rendered by said second pipeline based on said first coordinaterange, and wherein said second graphical pipeline is configured to receive an input identifying a second coordinate range and is configured to discard said graphical data rendered by said first graphical pipeline based on said second coordinate image.
4. The system of claim 1, wherein said compositor is configured to blend color values included in said graphical data rendered by said first and second graphical pipelines.
5. The system of claim 1, further comprising a graphics application, wherein each of the portions of said graphical data rendered by said plurality of graphical acceleration units is transmitted from said graphical application.
6. The system of claim 1, wherein said at least one graphical acceleration unit comprises an interface coupled to said first graphical pipeline via a first local area network (LAN) connection and coupled to said second graphical pipeline via asecond LAN connection, said interface of said at least one graphical acceleration unit configured to transmit said graphical command to said first and second graphical pipelines via said first and second LAN connections.
7. The system of claim 1, wherein said second graphical pipeline is configured to discard, without rendering, all graphical data in said graphical command.
8. The system of claim 1, wherein said graphical command defines an image to be displayed by said one of said plurality of display devices interfaced with said compositor, and wherein said graphical data rendered by said first graphical pipelineentirely defines said image to be displayed by said one display device interfaced with said compositor.
9. The system of claim 8, wherein said second graphical pipeline is configured to discard, without rendering, said graphical data from said graphical command.
10. The system of claim 8, wherein said second graphical pipeline is configured to render said graphical data from said graphical command.
11. The system of claim 1, wherein a compositor of another of said graphical acceleration units is configured to blend color values included in graphical data rendered by a graphical pipeline with color values included in graphical data renderedby another graphical pipeline.
12. A single logical screen (SLS) graphical display system, comprising: an interface configured to receive graphical data defining an image; a plurality of display devices; and a plurality of graphical acceleration units, each of saidplurality of graphical acceleration units respectively interfaced with one of said plurality of display devices and configured to render a portion of said graphical data to said one of said plurality of display devices such that said display devicesdisplay said image as a single logical screen, wherein at least one of said graphical acceleration units comprises: a first graphical pipeline configured to receive and process a graphical command, said first graphical pipeline configured to rendergraphical data from said graphical command and to mathematically combine a first offset with coordinate values included in said graphical data rendered by said first graphical pipeline; a second graphical pipeline configured to receive and process saidgraphical command, said second graphical pipeline configured to mathematically combine a second offset with coordinate values included in graphical data rendered by said second graphical pipeline; and a compositor interfaced with said first and secondgraphical pipelines and one of said display devices.
13. The system of claim 12, wherein said first and second graphical pipelines, by respectively combining said first and second offsets with coordinate values in said graphical data rendered by said first and second graphical pipelines, offsetsan image defined by said graphical data rendered by said first graphical pipeline with respect to an image defined by said graphical data rendered by said second graphical pipeline such that said compositor defines a jitter enhanced image by blendingsaid color values.
14. A single logical screen (SLS) graphical display system, comprising: first rendering means for rendering graphical data from a first graphical command, said first rendering means including a plurality of pipeline means for rendering, inparallel, said graphical data and a compositing means for compositing said graphical data, each of said plurality of pipeline means configured to super-sample graphical data from said first graphical command, wherein said compositing means includes ameans for blending color values included in said super-sampled graphical data; second rendering means for rendering graphical data from a second graphical command, said second rendering means including a plurality of pipeline means for rendering inparallel, said graphical data from said second graphical command and a compositing means for compositing said graphical data rendered by said second plurality of pipeline means; first display means for displaying a first image based on graphical datacomposited by said compositing means of said first rendering means; and second display means for displaying a second image based on graphical data composited by said compositing means of said second rendering means, wherein said first and second imagesdefine at least a portion of a single logical screen image.
15. The system of claim 14, wherein said first rendering means includes a means for receiving an input identifying a coordinate range, and wherein one of said plurality of pipeline means of said first rendering means includes a means fordiscarding, based on said coordinate range, graphical data from said first graphical command.
16. The system of claim 14, wherein said first rendering means comprises a first plurality of local network (LAN) connections, each of said first plurality of pipeline means configured to receive, from a different one of said first plurality ofLAN connections, a respective portion of said graphical data from said first graphical command, and wherein said second rendering means comprises a second plurality of local network (LAN) connections, each of said second plurality of pipeline meansconfigured to receive, from a different one of said second plurality of LAN connections, a respective portion of said graphical data from said second graphical command.
17. A single logical screen (SLS) graphical display system, comprising: first rendering means for rendering graphical data from a first graphical command, said first rendering means including a plurality of pipeline means for rendering, inparallel, said graphical data and a compositing means for compositing said graphical data, each of said plurality of pipeline means including a means for mathematically combining a different offset to coordinate values included in said graphical data; second rendering means for rendering graphical data from a second graphical command, said second rendering means including a plurality of pipeline means for rendering in parallel, said graphical data from said second graphical command and a compositingmeans for compositing said graphical data rendered by said second plurality of pipeline means; first display means for displaying a first image based on graphical data composited by said compositing means of said first rendering means; and seconddisplay means for displaying a second image based on graphical data composited by said compositing means of said second rendering means, wherein said first and second images define at least a portion of a single logical screen image.
18. A single logical screen (SLS) graphical display method, comprising: receiving graphical data defining an image; rendering different portions of said graphical data via different ones of a plurality of graphical acceleration units; in eachof said graphical acceleration units, compositing the graphical data rendered by said each graphical acceleration unit; and displaying said image across a plurality of display devices as a single logical screen, said displayed image based on saidcomposited graphical data, wherein said rendering comprises rendering, in one of said graphical acceleration units, graphical data from a single graphical command via each of a plurality of pipelines and super-sampling graphical data from said simplegraphical command, and wherein said compositing further comprises blending color values included in said super-sampled graphical data.
19. The method of claim 18, further comprising: receiving an input identifying a coordinate range; and discarding, via one of said plurality of graphical pipelines, graphical data from said single graphical command based on said coordinaterange.
20. The method of claim 18, further comprising transmitting each of said portions of said graphical data from a single graphics application.
21. The method of claim 18, further comprising transmitting, in said one graphical acceleration unit, graphical data from said single graphical command to each of said plurality of pipelines via a different local area network (LAN) connection.
22. The method of claim 18, wherein said rendering different portions comprises rendering the same image via each of a plurality of graphical pipelines in a single one of said graphical acceleration units.
23. The method of claim 22, wherein said rendering different portions comprises rendering, in parallel, different images via a plurality of graphical pipelines in a single one of said graphical acceleration units.
24. A single logical screen (SLS) graphical display method, comprising: receiving graphical data defining an image; rendering different portions of said graphical data via different ones of a plurality of graphical accelerating units; in eachof said graphical acceleration units, compositing the graphical data rendered by said each graphical acceleration unit; and displaying said image across a plurality of display devices as a single logical screen, said displayed image based on saidcomposited graphical data, wherein said rendering comprises rendering, in one of said graphical acceleration units, graphical data from a single graphical command via each of a plurality of pipelines and mathematically combining different offsets withcoordinate values included in said graphical data from said single graphical command.
25. The method of claim 24, wherein said combining causes said compositing to jitter enhance said image.
26. A single logical screen (SLS) graphical display system, comprising: an interface configured to receive graphical data defining an image; a plurality of display devices; and a plurality of graphical acceleration units, each of saidgraphical acceleration units interfaced with a respective one of said plurality of display devices and configured to render, in parallel, a different portion of said graphical data such that said display devices display said image as a single logicalscreen, each of said graphical acceleration units comprising a plurality of graphical pipelines and a compositor, wherein one of said graphical acceleration units is configured to render at least a portion of a three-dimensional graphical object, each ofa plurality of graphical pipelines of said one graphical acceleration unit configured to render, in parallel, at least a portion of said three-dimensional graphical object, and wherein the compositor of said one graphical acceleration unit is configuredto composite graphical data rendered by said plurality of graphical pipelines of said one graphical acceleration unit, wherein each of said plurality of graphical pipelines of said one graphical acceleration unit is configured to mathematically combine adifferent offset to corresponding coordinate values of graphical data defining said three-dimensional graphical object such that said compositor of said one graphical acceleration unit jitter enhances said three-dimensional graphical object.
27. The system of claim 26, wherein each of said plurality of graphical pipelines of said one graphical acceleration unit is configured to render a different portion of said three-dimensional graphical object.
28. The system of claim 26, wherein each of said graphical pipelines of said one graphical acceleration unit is configured to render and super-sample a different portion of said three-dimensional graphical object.
29. The system of claim 26, wherein said one graphical acceleration unit comprises an interface configured to transmit, to each of said plurality of graphical pipelines, each three-dimensional graphical command received by said one graphicalacceleration unit.
30. The system of claim 29, wherein said interface is coupled to each of said plurality of pipelines via a different local area network (LAN) connection.
31. The system of claim 26, wherein said one graphical acceleration unit comprises an interface configured to transmit, to each of said plurality of graphical pipelines, a plurality of three-dimensional graphical commands, wherein at least oneof said plurality of graphical pipelines is configured to discard, without rendering, all graphical data in one of said graphical commands.
32. The system of claim 31, wherein said interface is coupled to each of said plurality of pipelines via a different local area network (LAN) connection.
33. The system of claim 26, wherein each of a plurality of graphical pipelines of another of said graphical acceleration units is configured to render only a respective portion of an image to be displayed by a corresponding one of said displaydevices.
34. A single logical screen (SLS) graphical display method, comprising: receiving graphical data defining an image; displaying said image via a plurality of display devices as a single logical screen; for each of said display devices,rendering in parallel a different portion of said graphical data and compositing said rendered portion, wherein said rendering comprises rendering, in parallel for a single one of said display devices, at least a portion of a three-dimensional graphicalobject via a plurality of graphical pipelines and super-sampling said portion, and wherein said compositing comprises blending color values included in said super-sampled portion.
35. The method of claim 34, further comprising transmitting, to each of said graphical pipelines, each three-dimensional graphical command having graphical data to be rendered by said single one of said display devices.
36. A single logical screen (SLS) graphical display system, comprising: an interface configured to receive graphical data defining an image; a plurality of display devices; and a plurality of graphical acceleration units, each of saidplurality of graphical acceleration units respectively interfaced with one of said plurality of display devices and configured to render a portion of said graphical data to said one of said plurality of display devices such that said display devicesdisplay said image as a single logical screen, wherein at least one of said graphical acceleration units comprises: a first graphical pipeline configured to receive and process a graphical command, said first graphical pipeline configured to rendergraphical data from said graphical command and to mathematically combine a first offset with coordinate values included in said graphical data rendered by said first graphical pipeline; a second graphical pipeline configured to receive and process saidgraphical command, said second graphical pipeline configured to mathematically combine a second offset with coordinate values included in graphical data rendered by said second graphical pipeline; and a compositor interfaced with said first and secondgraphical pipelines and one of said display devices.
37. The system of claim 36, wherein each of a plurality of graphical pipelines of at least one of said graphical acceleration units is configured to render only a respectively portion of an image to be displayed by a corresponding one of saiddisplay devices. |
| Description: |
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention generally relates to techniques for rendering graphical data and, in particular, to an efficient system and method for rendering graphical data to a plurality of display devices that operate as a single logical screen.
2. Related Art
Computer graphical display systems are commonly used for displaying graphical representations of two-dimensional and/or three-dimensional objects on a two-dimensional display device, such as a cathode ray tube, for example. Current computergraphical display systems provide detailed visual representations of objects and are used in a variety of applications.
FIG. 1 depicts an exemplary embodiment of a conventional computer graphical display system 15. A graphics application 17 stored on a computer 21 defines, in data, an object to be rendered by the system 15. To render the object, the application17 transmit graphical data defining the object to graphics pipeline 23, which may be implemented in hardware, software, or a combination thereof. The graphics pipeline 23, through well known techniques, processes the graphical data receive from theapplication 17 and stores the graphical data in a frame buffer 26. The frame buffer 26 stores the graphical data necessary to define the image to be displayed by a display device 29. In this regard, the frame buffer 26 includes a set of data for eachpixel displayed by the display device 29. Each set of data is correlated with the coordinate values that identify one of the pixel displayed by the display device 29, and each set of data includes the color value of the identified pixel as well as anyadditional information needed to appropriately color or shade the identified pixel. Normally, the frame buffer 26 transmits the graphical data stored therein to the display device 29 via a scanning process such that each line of pixels defining theimage displayed by the display device 29 is consecutively updated.
When large images are to be displayed, multiple display devices may be used to display a single image, in which each display device displays a portion of the single image. In such an embodiment, the multiple display device are treated as asingle logical screen (SLS), and different portions of an object may be rendered by different display devices. FIG. 2 depicts an exemplary embodiment of a computer graphics system 41 capable of utilizing a plurality of display devices 31-34 to render asingle logical screen. In this embodiment, a client computer 42 stores the application 17 that defines, in data, an image to be displayed. Each of the display devices 31-34 may be used to display a portion of an object such that the display devices31-34, as a group, display a single large image of the object.
To render the object, graphical data defining the object is transmitted to an SLS server 45. The SLS server 45 routes the graphical data to each of the graphics pipelines 36-39 for processing and rendering. For example, assume that the objectis to be positioned such that each of the display devices 31-34 displays a portion of the object. Each of the pipeline 36-39 renders the graphical data into a form that can be written into one of the frame buffers 46-49. Once the data has been renderedby the pipelines 36-39 to the point that the graphical data is in a form suitable for storage into frame buffers 46-49, each of the pipelines 36-39 performs a clipping process before transmitting the data to frame buffers 46-49.
In the clipping process, each pipeline 36-39 discards the graphical data defining the portions of the object that are not to be displayed by the pipeline's associated display device 31-34 (i.e., the display device 31-34 coupled to the pipeline36-39 through one of the frame buffers 46-49). In other words, each graphics pipeline 36-39 discards the graphical data defining the portions of the object displayed by the display devices 31-34 that are not coupled to the pipeline 36-39 through one ofthe frame buffers 46-49. For example, pipeline 36 discards the graphical data defining the portions of the object that are displayed by display devices 32-34, and pipelines 37 discards the graphical data defining the portions of the object that aredisplayed by display device 31, 33 and 34.
Thus, each frame buffer 46-49 should only store the graphical data defining the portion of the object displayed by the display device 31-34 that is coupled to the frame buffer 46-49. At least one solution for providing SLS functionality in an XWindow System environment is taught by Jeffrey J. Walls, Ian A. Elliott, and John Marks in U.S. Pat. No. 6,088,005, filed Jan. 10, 1996, and entitled "Design and Method for a Large, Virtual Workspace," which is incorporated herein by reference.
A plurality of networked computer systems are often employed in implementing SLS technology. For example, in the embodiment shown by FIG. 2, the client 42, the SLS server 45, and the individual graphics pipelines 36-39 may each be implementedvia a single computer system interconnected with the other computer systems within the system 41 via a computer network, such a local area network (LAN), for example. The X Window System in a standard for implementing window-based user interfaces in anetworked computer environment, and it may be desirable to utilize X Protocol in rendering graphical data in the system 41. For a more detailed discussion of the X Window System and the X Protocol that defines its, see Adrian Nye, X Protocol ReferenceManual Volume Zero (O'Riley & Associates 1990).
U.S. patent application Ser. No. 09/138,456, filed on Aug. 21, 1998, and entitled "3D Graphics in a Single Logical Screen Display Using Multiple Remote Computer Systems," which is incorporated herein by reference, describes an SLS system ofnetworked computer stations that may be used to render two-dimensional (2D) and three-dimensional (3D) graphical data. In the embodiments described by the foregoing patent application, X Protocol is generally utilized to render 2D graphical data, andOpenGL Protocol (OGL) is generally used to render 3D graphical data.
Although it is possible to render 2D and/or 3D data in conventional computer graphical display systems, including SL environments, there exists limitations that restrict the performance and/or image quality exhibited by the conventional computergraphical display systems. More specifically, high quality images, particularly 3D images, are typically defined by a large amount of graphical data, and the speed at which conventional graphics pipelines 36-39 can process the graphical data defining anobject is limited. Thus, a trade-off often exists between increasing the quality of the image rendered by a computer graphical display system and the speed at which the image can be rendered, and there exists a need in the industry for better techniquesand systems for rendering graphical data.
SUMMARY OF THE INVENTION
The present invention overcomes the inadequacies and deficiencies of the prior art as discussed hereinbefore. Generally, the present invention provides a graphical display system and method for efficiently rendering graphical data to display animage as a single logical screen.
In architecture, the graphical display system of the present invention utilizes a plurality of display devices and a plurality of graphic acceleration units for rendering graphical data to the display devices. More specifically, each of theplurality of graphical acceleration units respectively interfaces a portion of graphical data defining an image to one of the display devices. Each of the display devices displays a portion of the image based on the graphical data rendered to it. Tomake the system more efficient and/or to improve image quality, at least one of the graphical acceleration units includes a plurality of graphical pipelines for rendering the graphical data to be displayed by the display device that is interfaced withthe one graphical acceleration unit.
The present invention can also be viewed as providing a single logical screen (SLS) method for displaying graphical images. The method can be broadly conceptualized by the following steps: providing a plurality of graphical acceleration units,at least one of the graphical acceleration units including a plurality of graphical pipelines; providing a plurality of display devices; receiving graphical data defining an image; rendering different portions of the graphical data via different ones ofthe graphical acceleration units; and displaying the image across the display devices as a single logical screen based on the graphical data rendered in the rendering step, wherein the rendering step includes the step of rendering graphical data from oneof the portions via each of the plurality of pipelines.
Other features and advantages of the present invention will become apparent to one skilled in the art upon examination of the following detailed description, when read in conjunction with the accompanying drawings. It is intended that all suchfeatures and advantages be included herein within the scope of the present invention and protected by the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention can be better understood with reference to the following drawings. The elements of the drawings are not necessarily to scale relative to each other, emphasis instead being placed upon clearly illustrating the principles of theinvention. Furthermore, like reference numerals designate corresponding parts throughout the several views.
FIG. 1 is a block diagram illustrating a conventional graphical display system.
FIG. 2 is a block diagram illustrating a conventional single logical screen (SLS) graphical display system.
FIG. 3 is a block diagram illustrating a graphical display system in accordance with the present invention.
FIG. 4 is a block diagram illustrating a more detailed view of a client depicted in FIG. 3.
FIG. 5 is a block diagram illustrating a more detailed view of a master pipeline depicted in FIG. 3.
FIG. 6 is a block diagram illustrating a more detailed view of a slave pipeline depicted in FIG. 3.
FIG. 7 is a diagram illustrating a more detailed view of a display device depicted in FIG. 3. The display device of FIG. 7 is displaying an exemplary X window having a center region for displaying three-dimensional objects.
FIG. 8 is a diagram illustrating the display device depicted in FIG. 7 with the center region partitioned according to one embodiment of the present invention.
FIG. 9 is a diagram illustrating the display device depicted in FIG. 7 with the center region partitioned according to another embodiment of the present invention.
FIG. 10 is a diagram illustrating the display device depicted in FIG. 8 with a three-dimensional object displayed within the center region.
FIG. 11 is a diagram illustrating the display device depicted in FIG. 7 when super-sampled data residing in one of the frame buffers interfaced with one of the slave pipelines is displayed within the center region of the display device.
FIG. 12 is a diagram illustrating the display device depicted in FIG. 11 when super-sampled data residing in another of the frame buffers interfaced with another of the slave pipelines is displayed within the center region of the display device.
FIG. 13 is a block diagram illustrating another embodiment of the graphical display system depicted in FIG. 3.
FIG. 14 is a single logical screen (SLS) graphical display system that utilizes a graphical acceleration unit depicted in FIG. 3 or FIG. 13.
FIG. 15 is a diagram illustrating a more detailed view of display devices that are depicted in FIG. 14.
FIG. 16 is a block diagram illustrating a graphical display system in accordance with the present invention.
FIG. 17 is a flowchart illustrating functionality of a preferred embodiment of the compositor of the present invention.
FIG. 18 is a flowchart illustrating functionality of a preferred compositor of the present invention.
FIG. 19 is a block diagram illustrating a portion of a preferred embodiment of the compositor of the present invention.
FIG. 20A is a schematic diagram illustrating a representative active stereo system which may be implemented by a preferred embodiment of the present invention.
FIG. 20B is a schematic diagram illustrating the representative active stereo system of FIG. 20A.
FIG. 21 is a block diagram illustrating a portion of a preferred embodiment of the compositor of the present invention.
FIG. 22 is a diagram illustrating a representative frame buffer sequence in relation to an image sequence corresponding to an active stereo implementation of the present invention.
FIG. 23 is a schematic diagram illustrating a representative passive stereo system which may be implemented a preferred embodiment of the present invention.
FIG. 24 is a diagram illustrating a representative frame buffer sequence in relation to an image sequence corresponding to a passive stereo implementation of the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
In general, the present invention pertains to a computer graphical display system employing a plurality of graphics pipelines that process graphical data in parallel. The graphics pipelines provide the graphical data to a compositor thatcombines or composites the graphical data into a single data stream that can be rendered via a single display device. The parallel processing of graphical data according to the present invention enables improved performance and/or improved image qualityover conventional computer graphical display systems.
FIG. 3 depicts a computer graphical display system 50 in accordance with the present invention. As shown by FIG. 3, the system 50 includes a client 52, a master graphics pipeline 55, and one or more slave graphics pipelines 56-59. The client 52and pipelines 55-59 may be implemented via hardware, software or any combination thereof. It should be noted that the embodiment shown by FIG. 3 depicts four slave pipelines 56-59 for illustrative purposes only, and any number of slave pipelines 56-59may be employed to implement the present invention in other embodiments. As shown by FIG. 3, the pipelines 55-59, frame buffers 65-69, and compositor 76 that render graphical data to a single display device 83 are collectively referred to herein as agraphical accelerations unit 95.
The master pipeline 55 receives graphical data from the application 17 stored in the client 52. The master pipeline 55 preferably renders two-dimensional (2D) graphical data to frame buffer 65 and routes three-dimensional (3D) graphical data toslave pipelines 56-59, which render the 3D graphical data to frame buffers 66-69, respectively. Except as otherwise described herein, the client 52 and the pipelines 55-59 may be configured similar to pipelines described in U.S. patent application Ser. No. 09/138,456. The client 52 and the pipelines 55-59 will be described in more detail hereinafter.
Each frame buffer 65-69 outputs a stream of graphical data to the compositor 76. The compositor 76 is configured to combine or composite each of the data streams from frame buffers 65-69 into a single data stream that is provided to displaydevice 83, which may be a monitor (e.g., cathode ray tube) or other device for displaying an image. The graphical data provided to the display device 83 by the compositor 76 defines the image to be displayed by the display device 83 and is based on thegraphical data received from frame buffers 65-69. The compositor 76 will be further described in more detail hereinafter. Note that each data stream depicted in FIG. 3 may be either a serial data stream or a parallel data stream.
In the preferred embodiment, the client 52 and each of the pipelines 55-59 are respectively implemented via stand alone computer systems, commonly referred to as a "computer workstations." Thus, the system 50 shown by FIG. 3 may be implementedvia six computer workstations (i.e., one computer workstation for the client 52 and one computer workstation for each of the pipelines 55-59). However, if is possible to implement the client 52 and pipelines 55-59 via other configurations, includingother numbers of computer workstations or no computer workstations. As an example, the client 52 and the master pipeline 55 may be implemented via a single computer workstation. Any computer workstation used to implement the client 52 and/or pipelines55-59 may be utilized to perform other desired functionality when the workstation is not being used to render graphical data.
Furthermore, as shown by FIG. 3, the client 52 and the pipelines 55-59 may be interconnected via a local area network (LAN) 62. However, it is possible to utilize other types of interconnection circuitry without departing from the principles ofthe present invention.
FIG. 4 depicts a more detailed view of the client 52. As can be seen by referring to FIG. 4, the client 52 preferably stores the graphics application 17 in memory 102. Through conventional techniques, the application 17 is executed by anoperating system 105 and one or more conventional processing elements 111, such as a central processing unit (CPU), for example. The operating system 105 performs functionality similar to conventional operating systems. More specifically, the operatingsystem 105 controls the resources of the client 52 through conventional techniques and interfaces the instructions of the application 17 with the processing element 111 as necessary to enable the application 17 to run properly.
The processing element 111 communicates to and drives the other elements within the client 52 via a local interface 113, which can include one or more buses. Furthermore, an input device 115, for example, a keyboard or a mouse, can be used toinput data from a user of the client 52, and an output device 117, for example, a display device or a printer, can be used to output data to the user. A disk storage mechanism 122 can be connected to the local interface 113 to transfer data to and froma nonvolatile disk (e.g., magnetic, optical, etc.). The client 52 is preferably connected to a LAN interface 126 that allows the client 52 to exchange data with the LAN 62.
In the preferred embodiment, X Protocol is generally utilized to render 2D graphical data, and OpenGL Protocol (OGL) is generally utilized to render 3D graphical data, although other types of protocols may be utilized in other embodiments. Byway of background, OpenGL Protocol is a standard application programmer's interface (API) to hardware that accelerates 3D graphics operations. Although OpenGL Protocol is designed to be window system independent, it is often used with window systems,such as the X Window System, for example. In order that OpenGL Protocol may be used in an X Window System environment, an extension of the X Window System has been developed called GLX. For more complete information on the GLX extension to the X WindowSystem and on how OpenGL Protocol can be integrated with the X Window System, see for example Mark J. Kilgard, OpenGL Programming for the X Window System (Addison-Wesley Developers Press 1996), which is incorporated herein by reference.
When the application 17 issues a graphical command, a client side GLX layer 131 of the client 52 transmits the command over LAN 62 to master pipeline 55. FIG. 5 depicts a more detailed view of the master pipeline 55. Similar to client 52, themaster pipeline 55 includes one or more processing elements 141 that communicate to and drive the other elements within the master pipeline 55 via a local interface 143, which can include one or more buses. Furthermore, an input device 145, for example,a keyboard or a mouse, can be used to input data from a user of the pipeline 55, and an output device 147, for example, a display device or a printer, can be used to output data to the user. A disk storage mechanism 152 can be connected to the localinterface 143 to transfer data to and from a nonvolatile disk (e.g., magnetic, optical, etc.). The pipeline 55 may be connected to a LAN interface 156 that allows the pipeline 55 to exchange data with the LAN 62.
The pipeline 55 also includes an X server 162. The X server 162 may be implemented in software, hardware, or a combination thereof, and in the embodiment shown by FIG. 5, the X server 162 is implemented in software and stored in memory 162. Inthe preferred embodiment, the X server 162 renders 2D X window commands, such as commands to create or move an X window. In this regard, an X server dispatch layer 173 is designed to route received commands to a device independent layer (DIX) 175 or toa GLX layer 177. An X window command that does not include 3D data is interfaced with DIX, whereas an X window command that does include 3D data (e.g., an X command having embedded OGL protocol, such as a command to create or change the state of a 3Dimage within an X window) is routed to GLX layer 177. A command interfaced with the DIX 175 is executed by the DIX 175 and potentially by a device dependent layer (DDX) 179, which drives graphical data associated with the executed command throughpipeline hardware 166 to frame buffer 65. A command interfaced with GLX layer 177 is transmitted by the GLX layer 177 across LAN 62 to slave pipelines 56-59. One or more of the pipelines 56-59 executes the command and drives graphical data associatedwith the command to one or more frame buffers 66-69.
In the preferred embodiment, each of slave pipelines 56-59 is configured according to FIG. 6, although other configurations of pipelines 56-59 in other embodiments are possible. As shown by FIG. 6, each slave pipeline 56-59 includes an X server202, similar to the X server 162 previously described, and an OGL daemon 205. The X server 202 and OGL daemon 205 may be implemented in software, hardware, or a combination thereof, and in the embodiment shown by FIG. 6, the X server 202 and OGL daemon205 are implemented in software and stored in memory 206. Similar to client 52 and master pipeline 55, each of the slave pipelines 56-59 includes one or more processing elements 181 that communicate to an drive the other elements within the pipeline56-59 via a local interface 183, which can include one or more buses. Furthermore, an input device 185, for example, a keyboard or a mouse, can be used to input data from a user of the pipeline 56-59, and an output device 187, for example, a displaydevice or a printer, can be used to output data to the user. A disk storage mechanism 192 can be connected to the local interface 183 to transfer data to and from a nonvolatile disk (e.g., magnetic, optical, etc.). Each pipeline 56-59 is preferablyconnected to a LAN interface 196 that allows the pipeline 56-59 to exchange data with the LAN 62.
Similar to X server 162, the X server 202 includes an X server dispatch layer 208, a GLX layer 211, a DIX layer 214, and a DDX layer 216. In the preferred embodiment, each command received by the slave pipelines 56-59 includes 3D graphical data,since the X server 162 of master pipeline 55 executes each X window command that does not include 3D graphical data. The X server dispatch layer 208 interfaces the 2D data of any received commands with DIX layer 214 and interfaces the 3D data of anyreceived commands with GLX layer 211. The DIX and DDX layers 214 and 216 are configured to process or accelerate the 2D data and to drive the 2D data through pipeline hardware 166 to one of the frame buffers 66-69 (FIG. 3).
The GLX layer 211 interfaces the 3D data with the OGL dispatch layer 223 of the OGL daemon 205. The OGL dispatch layer 223 interfaces this data with the OGL DI layer 225. The OGL DI layer 225 and DD layer 227 are configured to process the 3Ddata and to accelerate or drive the 3D data through pipeline hardware 199 to one of the frame buffers 66-69 (FIG. 3). Thus, the 2D graphical data of a received command is processed or accelerated by the X server 202, and the 3D graphical data of thereceived command is proceeded or accelerated by the OGL daemon 205. For a more detailed description of the foregoing process of accelerating 2D data via an X server 202 and of accelerating 3D data via an OGL daemon 205, refer to U.S. patent applicationSer. No. 09/138,456.
Preferably, the slave pipelines 56-59, based on inputs from the master pipeline 55, are configured to render 3D images based on the graphical data from master pipeline 55 according to one of three modes of operation: the optimization mode, thesuper-sampling mode, and the jitter mode. In the optimization mode, each of the slave pipelines 56-59 renders a different portion of a 3D image such that the overall process of rendering the 3D image is faster. In the supper-sampling mode, each portionof a 3D image rendered by one or more of the slave pipelines 56-59 is super-sampled in order to increase quality of the 3D image via anti-aliasing. Furthermore, in the jitter mode, each of the slave pipelines 56-59 renders the same 3D image but slightlyoffsets each rendered 3D image with a different offset value. Then, the compositor 76 averages the pixel data of each pixel for the 3D images rendered by the pipelines 56-59 in order to produce a single 3D image of increased image quality. Each of theforegoing modes will be described in more detail hereafter.
Optimization Mode
Referring to FIG. 3, the operation and interaction of the client 52, the pipelines 55-59, and the compositor 76 will now be described in more detail according to a preferred embodiment of the present invention while the system 50 is operating inthe optimization mode. In such an embodiment, the master pipeline 55, in addition to controlling the operation of the slave pipelines 56-59 as described hereinafter, is used to create and manipulate an X window to be displayed by the display device 83. Furthermore, each of the slave pipelines 56-59 is used to render 3D graphical data within a portion of the foregoing X window.
For the purposes of illustrating the aforementioned embodiment, assume that the application 17 issues a function call (i.e., the client 52 via processing element 111 (FIG. 4) executes a function call within the application 17) for creating an Xwindow having a 3D image displayed within the X window. FIG. 7 depicts a more detailed view of the display device 83 displaying such a window 245 on a display device screen 247. In the example shown by FIG. 7, the screen 247 is 2000 pixels by 2000pixels ("2K.times.2K"), and the X window 245 is 1000 pixels by 1000 pixels ("1K.times.1K"). The window 245 is offset from each edge of the screen 247 by 500 pixels. Assume that 3D graphical data is to be rendered in a center region 249 of the X window245. This center region 249 is offset from each of the window 245 by 200 pixels in the embodiment shown by FIG. 7.
In response to execution of the function call by client 52, the application 17 transmit to the master pipeline 55 a command to render the X window 245 and a command to render a 3D image within portion 249 of the X window 245. The command forrendering the X window 245 should include 2D graphical data defining the X window 245, and the command for rendering the 3D image within the X window 245 should include 3D graphical data defining the 3D image to be displayed within region 249. Preferably, the master pipeline 55 renders 2D graphical data from the former command (i.e., the command for rendering the X window 245) to frame buffer 65 (FIG. 3) via X server 162 (FIG. 6).
The graphical data rendered by any of the pipelines 55-59 includes sets of values that respectively define a plurality of pixels. Each set of values includes at least a color value and a plurality of coordinate values associated with the pixelbeing defined by the set of values. The coordinate values define the pixel's position relative to the other pixels defined by the graphical data, and the color value indicates how the pixel should be colored. While the coordinate values indicate thepixel's position relative to the other pixels defined by the graphical data, the coordinate values produced by the application 17 are not the same coordinate values assigned by the display device 83 to each pixel of the screen 247. Thus, the pipelines55-59 should translate the coordinate values of each pixel rendered by the pipelines 55-59 to the coordinate values used by the display device 83 to display images. Sometimes the coordinate values produced by the application 17 are said to be "windowrelative," and the aforementioned coordinate values translated from the window relative coordinates are said to be "screen relative." The concept of translating window relate coordinates to screen relative coordinates is well known, and techniques fortranslating window relative coordinates to screen relative coordinates are employed by most conventional graphical display systems.
In addition to translating coordinates of the 2D data rendered by the master pipeline 55 from window relative to screen relative, the master pipeline 55 in each mode of operation also assigns a particular color value, referred to hereafter as the"chroma-key," to each pixel within the region 249. The chroma-key indicates which pixels within the X window 245 may be assigned a color value of a 3D image that is generated by slave pipelines 56-59. In this regard, each pixel assigned the chroma-keyas the color value by master server 55 is within region 249 and, therefore, may be assigned a color of a 3D object rendered by slave pipelines 56-59, as will be described in further detail hereafter. In the example shown by FIG. 7, the graphical datarendered by master pipeline 55 and associated with screen relative coordinate values ranging from (700, 700) to (1300, 1300) are assigned the chroma-key as their color value by the master pipeline 55, since the region 249 is the portion of X window 245that is to be used for displaying 3D images.
As shown by FIG. 5, the master pipeline 55 includes a slave controller 261 that is configured to provide inputs to each slave pipeline 56-59 over the LAN 62. The slave controller 261 may be implemented in software, hardware, or a combinationthereof, and in the embodiment shown by FIG. 5, the slave controller 261 is implemented in software and stored in memory 206. The inputs from the slave controller 261 inform the slaves 56-59 of which mode each slave 56-59 should presently operate. Inthe present example, the slave controller 261 transmits inputs to each slave 56-59 indicating that each slave 56-59 should be in the optimization mode of operation. The inputs from slave controller 261 also indicate which portion of region 249 (FIG. 7)that is each slave's responsibility. For example, assume for illustrative purposes, that each slave 56-59 is responsible for rendering the graphical data displayed in one of the portions 266-269 shown by FIG. 8.
In this regard, assume that: (1) slave pipeline 56 is responsible for rendering graphical data defining the image displayed in portion 266 (i.e., screen relative coordinates (700, 1000) to (1000, 1300), (2) slave pipeline 57 is responsible forrendering graphical data defining the image displayed in portion 267 (i.e., screen relative coordinates (1000, 1000) to (1300, 1300), (3) slave pipeline 58 is responsible for rendering graphical data defining the image displayed in portion 268 (i.e.,screen relative coordinates (700, 700) to (1000, 1000), and (4) slave pipeline 59 is responsible for rendering graphical data defining the image displayed in portion 269 (i.e., screen relative coordinates (1000, 700) to (1300, 1000). The inputstransmitted by the slave controller 261 to the slave pipelines 56-59 preferably indicate the range of screen coordinate values that each slave pipeline 56-59 is responsible for rendering.
Note that the partition of the region 249 can be divided among the pipelines 56-59 via other configurations, and it is not necessary for each pipeline 56-59 to be responsible for an equally sized area of the region 249. For example, FIG. 9 showsan embodiment where each portion 266-269 represents a different sized horizontal area of the region 249.
Each slave pipeline 56-59 is configured to receive from master pipeline 55 the graphical data of the command for rendering the 3D image to be displayed in region 249 and to render this data to frame buffers 66-69, respectively. In this regard,each pipeline 56-59 renders graphical data defining a 2D X window that displays a 3D image within the window. More specifically, slave pipeline 56 renders graphical data to frame buffer 66 that defines an X window displaying a 3D image within portion266 (FIG. 8). The X server 202 within slave pipeline 56 renders the data that defines the foregoing X window, and the OGL daemon 205 within the slave pipeline 56 renders that data that defines the 3D image displayed within the foregoing X window. Furthermore, slave pipeline 57 renders graphical data to frame buffer 67 that defines an X window displaying a 3D image within portion 267 (FIG. 8). The X server 202 within slave pipeline 57 renders the data that defines the foregoing X window, and theOGL daemon 205 within the slave pipeline 57 renders the data that defines the 3D images displayed within the foregoing X window. Similarly, slave pipelines 58 and 59 render graphical data to frame buffers 68 and 69, respectively, via the X server 202and the OGL daemon 205 within the pipelines 58 and 59.
Note that the graphical data rendered by each pipeline 56-59 defines a portion of the overall image to be displayed within region 249. Thus, it is not necessary for each pipeline 56-59 to render all of the graphical data defining the entire 3Dimage to be displayed in region 249. Indeed, in the preferred embodiment, each slave pipeline 56-59 preferably discards the graphical data that defines a portion of the image that is outside of the pipeline's responsibility. In this regard, eachpipeline 56-59 receives from master pipeline 55 the graphical data that defines the 3D image to be displayed in region 249. Each pipeline 56-59, based on the aforementioned inputs received from slave controller 261 then determines which portion of thisgraphical data is within pipeline's responsibility and discards the graphical data outside of this portion.
For example, as described previously, slave pipeline 56 is responsible for rendering the graphical data defining the image to be displayed within portion 266 of FIG. 8. This portion 266 includes graphical data associated with screen relativecoordinates (700, 1000) to (1000, 1300). Thus, any graphical data having screen relative coordinates outside of this range is discarded by the pipeline 56, and only graphical data having screen relative coordinates within the foregoing range is renderedto frame buffer 66.
Furthermore, slave pipeline 57 is responsible for rendering the graphical data defining the image to be displayed within portion 267 of FIG. 8. This portion 267 includes graphical data associated with screen relative coordinates (1000, 1000) to(1300, 1300). Thus, any graphical data having screen relative coordinates outside of this range is discarded by the pipeline 57, and only graphical data having screen relative coordinates within the foregoing range is rendered to frame buffer 67.
In addition, slave pipeline 58 is responsible for rendering the graphical data defining the image to be displayed within portion 268 of FIG. 8. This portion 268 includes graphical data associated with screen relative coordinates (700, 700) to(1000, 1000). Thus, any graphical data having screen relative coordinates outside of this range is discarded by the pipeline 58, and only graphical data having screen relative coordinates within the foregoing range is rendered to frame buffer 68.
Also, slave pipeline 59 is responsible for rendering the graphical data defining the image to be displayed within portion 269 of FIG. 8. This portion 269 includes graphical data associated with screen relative coordinates (1000, 700) to (1300,1000). This, any graphical data having screen relative coordinates outside of this range is discarded by the pipeline 59, and only graphical data having screen relative coordinates within the foregoing range is rendered to frame buffer 69.
To increase the efficiency of the system 50, each slave pipeline 56-59 preferably discards the graphical data outside of the pipeline's responsibility before significantly processing any of the data to be discarded. Bounding box techniques maybe employed to enable each pipeline 56-59 to quickly discard a large amount of graphical data outside of the pipeline's responsibility before significantly processing such graphical data.
In this regard, each set of graphical data transmitted to pipelines 56-59 may be associated with a particular set of bounding box data. The bounding box data defines a graphical bounding box that contains at least each pixel included in thegraphical data that is associated with the bounding box data. The bounding box data can be quickly processed and analyzed to determined whether a pipeline 56-59 is responsible for rendering any of the pixels included within the bounding box. If thepipeline 56-59 is responsible for rendering any of the pixels included within the bounding box, then the pipeline 56-59 renders the received graphical data that is associated with the bounding box. However, if the pipeline 56.59 is not responsible forrendering ay of the pixels included within the bounding box, then the pipeline 56.59 discards the received graphical data that is associated with the bounding box, and the pipeline 56.59 does not attempt to render the discarded graphical data. Thus,processing power is not wasted in rendering any graphical data that defines an object outside of the pipeline's responsibility and that can be discarded via the utilization of bounding box techniques as described above. Bounding box techniques are morefully described in U.S. Pat. No. 5,757,321, entitled "Apparatus and Method for Clipping Primitives Using Information from a Previous Bounding Box Process," which is incorporated herein by reference.
After the pipelines 56-59 have respectively rendered graphical data to frame buffers 65-69, the graphical data is read out of frame buffers 65-69 through conventional techniques and transmitted to compositor 76. Through techniques described inmore detail hereafter, the compositor 76 is designed to composite or combine the data streams from frame buffers 65-69 into a single data stream and to render the data from this single data stream to display device 83.
Once the graphical data produced by the application 17 has been rendered to display device 83, as described above, the display device 83 should display an image defined by the foregoing graphical data. This image may be modified by rendering newgraphical data from the application 17 via the same techniques described hereinabove. For example, assume that it is desirable to display a new 3D object 284 on the screen 247, as shown by FIG. 10. In this example, assume that an upper half of theobject 284 is to be displayed in the portion 266 and that a bottom half of the object is to be displayed in the portion 268. Thus, the object is not to be displayed in portion 267 and 269.
In the foregoing example, graphical data defining the object 284 is transmitted from client 52 to master pipeline 55. The master pipeline 55 transmits this graphical data to each of the slave pipelines 56-59. Since the object 284 is not to bedisplayed within portions 267 and 269, the screen coordinates of the object 284 should be outside of the ranges rendered by pipelines 57 and 59. Thus, slave pipelines 57 and 59 should discard the graphical data without rendering it to frame buffers 67and 69. Preferably, bounding box techniques and/or other data optimization techniques are employed to discard the graphical data defining the object 248 before the coordinates of this graphical data are translated to screen relative to by pipelines 57and 59 and/or before other significant processing is performed on this data by pipelines 57 and 59.
Since the top half of the object 284 is to be displayed within portion 266, the screen coordinates of the object should be within the range rendered by pipeline 56 (i.e., from screen coordinates (700, 1000) to (1000, 1300)). Thus, slave pipeline56 should render the graphical data defining the top half of the object 284 to frame buffer 66. However, since the bottom half of the object 284 is not to be displayed within portion 266, the screen coordinates of the bottom half of the object 284should be outside of the range rendered by the pipeline 56. Thus, the slave pipeline 56 should discard the graphical data defining the bottom half of the object 284 without rendering this data to frame buffer 66. Preferably, bounding box techniquesand/or other data optimization techniques are employed to discard the graphical data defining the bottom half of the object 284 before the coordinates of this graphical data are translated to screen relative by pipeline 56 and/or before other significantprocessing is performed on this data by pipeline 56.
Since the bottom half of the object 284 is to be displayed within portion 268, the screen coordinates of the object should be within the range rendered by pipeline 58 (i.e., from screen coordinates (700, 700) to (1000, 1000)). Thus, slavepipeline 58 should render the graphical data defining the bottom half of the object 284 to frame buffer 68. However, since the top half of the object 284 is not to be displayed within portion 268, the screen coordinates of the top half of the object 284should be outside of the range rendered by the pipeline 58. Thus, the slave pipeline 58 should discard the graphical data defining the top half of the object 284 without rendering this data to frame buffer 68. Preferably, bounding box techniques and/orother data optimization techniques are employed to discard the graphical data defining the top half of the object 284 before the coordinates of this graphical data are translated to screen relative by pipeline 58 and/or before other significantprocessing is performed on this data by pipeline 58.
As described hereinbefore, the graphical data stored in frame buffers 65-69 should be composited by compositor 76 and rendered to display device 83. The display device 83 should then update the image displayed by the screen 247 such that theobject 284 is displayed within portions 266 and 268, as shown by FIG. 10.
Since each pipeline 55-59 renders only a portion of the graphical data defining each image displayed by display device 83, the total time for rendering the graphical data to display device 83 can be significantly decreased, thereby resulting inincreased efficiency for the system 50. Thus, in the optimization mode, the speed at which graphical data is rendered from the client 52 to the display device 83 should be maximized. This increase in efficiency is transparent to the application 17, inthat the application 17 does not need to be aware of the configuration of the pipeline 55-59 to operate correctly. Thus, the application 17 does not need to be modified to operate successfully in either conventional system 15 or in the system 50depicted by FIG. 3.
Super-Sampling Mode
Referring to FIG. 3, the operation and interaction of the client 52, pipelines 55-59, and the compositor 76 will now be described in more detail while each of the pipelines 56-59 is operating in the supper-sampling mode. In the super-samplingmode, the graphical data transmitted from the client 52 is super-sampled to enable anti-aliasing of the image produced by display device 83.
For illustrative purposes assume that the application 17, as described hereinabove for the optimization mode, issues a function call for creating an X window 245 having a 3D image displayed within the region 249 of the X window 245, as shown byFIG. 7. In the super-sampling mode, the pipelines 55-59 perform the same functionality as in the optimization mode except for a few differences, which will be described in more detail hereinbelow. More specifically, the client 52 transmits to themaster pipeline 55 a command to render the X window 245 and a command to render a 3D image within portion 249 of the X window 245. The command for rendering the X window 245 should include 2D graphical data defining the X window 245, and the command forrendering the 3D image within the X window 245 should include 3D graphical data defining the 3D image to be displayed within region 249. The master pipeline 55 renders the 2D data defining the X window 245 to frame buffer 65 and transmits the 3D datadefining the 3D image to slave pipelines 56-59, as described hereinabove for the optimization mode. The master pipeline 55 also assigns the chroma-key to each pixel that is rendered to frame buffer 65 and that is within portion 249.
The slave controller 261 transmits inputs to the slave pipelines 56-59 indicating the range of screen coordinate values that each slave 56-59 is responsible for rendering, as described hereinabove for the optimization mode. Each slave pipeline56-59 discards the graphical data outside of the pipeline's responsibility, as previously described for the optimization mode. However, unlike in the optimization mode, the pipelines 56-59 super-sample the graphical data rendered by the pipeline 56-59to frame buffers 66-69, respectively. In super-sampling the graphical data, the number of pixels used to represent the image defined by the graphical data is increased. Thus, a portion of the image represented as a single pixel in the optimization modeis instead represent as multiple pixels in the super-sampling mode. In other words, the image defined by the super-sampled data is blown up or magnified as compared to the image defined by the data prior to super-sampling. The graphical datasuper-sampled by pipelines 56-59 is rendered to frame buffers 66-69, respectively.
The graphical data stored in frame buffers 65-69 is then transmitted to compositor 76, which then combines or composites the graphical data into a single data stream for display device 83. Before compositing or combining the graphical data, thecompositor 76 first processes the super-sampled data received from frame buffers 66-69. More specifically, the compositor 76 reduces the size of the image defined by the super-sampled data back to the size of the image prior to the super-samplingperformed by pipelines 56-59. In reducing the size of the image defined by the super-sampled data, the compositor 76 averages or blends the color values of each set of super-sampled pixels that is reduced to a single pixel such that the resulting imagedefined by the processed data is anti-aliased.
As an example, assume that a portion of the graphical data originally defining a single pixel is super-sampled by one of the pipelines 56-59 into four pixels. When the foregoing portion of the graphical data is processed by compositor 76, thefour pixels are reduced to a single pixel having a color value that is an average or a blend of the color values of the four pixels. By performing the super-sampling and blending for each pixel defined by the graphical data transmitted to pipelines56-59, the entire image defined by this data is anti-aliased. Note that super-sampling of the single pixel into four pixels as described above is exemplary, and the single pixel may be super-sampled into numbers of pixels other than four in otherexamples. Further, any conventional technique and/or algorithm for blending pixels to form a jitter enhanced image may be employed by the compositor 76 to improve the quality of the image defined by the graphical data stored within frame buffers 66-69.
To better illustrate the operation of the system 50 in the super-sampling mode, assume that the application 17 issues a command to display the 3D object 284 depicted in FIG. 10. In the this example, graphical data defining the object 284 istransmitted from client 52 to master pipeline 55. The master pipeline 55 transmits this graphical data to each of the slave pipelines 56-59. Since the object 284 is not to be displayed within portions 267 and 269, the screen coordinates of the object284 should be outside of the ranges rendered by pipelines 57 and 59. Thus, slave pipelines 57 and 59 should discard the graphical data without rendering it to frame buffers 67 and 69. Preferably, bounding box techniques and/or other data optimizationtechniques are employed to discard the graphical data defining the object 284 before the coordinates of this graphical data are translated to screen relative by pipelines 57 and 59 and/or before other significant processing is performed on this data bypipelines 57 and 59.
Since the top half of the object 284 is to be displayed within portion 266, the screen coordinates of the object should be within the range rendered by pipeline 56 (i.e., from screen coordinates (700, 1000) to (1000, 1300)). Thus, slave pipeline56 should render the graphical data defining the tap half of the object 284 to frame buffer 66. However, since the bottom half of the object 284 is not to be displayed within portion 266, the screen coordinates of the bottom half of the object 284should be outside of the range rendered by the pipeline 56. Thus, the slave pipeline 56 should discard the graphical data defining the bottom half of the object 284 without rendering this data to frame buffer 66. Preferably, bounding box techniquesand/or other data optimization techniques are employed to discard the graphical data defining the bottom half of the object 284 before the coordinates of this graphical data are translated to screen relative by pipeline 56 and/or before other significantprocessing is performed on this data by pipeline 56.
In rendering the top half of the object 284, the pipeline 56 super-samples the data defining the top half of object 284 before storing this data in frame buffer 66. For illustrate purposes, assume that each pixel defining the top half of object284 is super-sampled by pipeline 56 into four pixels. Thus, if the super-sampled data stored in frame buffer 66 were somehow directly rendered in region 249 without the processing performed by compositor 76, the image displayed by display device 83should appear to be magnified as shown in FIG. 11.
Since the bottom half of the object 284 is to be displayed within portion 268, the screen coordinates of the object should be within the range rendered by pipeline 58 (i.e., from screen coordinates (700, 700) to (1000, 1000)). Thus, slavepipeline 58 should render the graphical data defining the bottom half of the object 284 to frame buffer 68. However, since the top half of the object 284 is not to be displayed within portion 268, the screen coordinates of the top half of the object 284should be outside of the range rendered by the pipeline 58. Thus, the slave pipeline 58 should discard the graphical data defining the top half of the object 284 without rendering this data to frame buffer 68. Preferably, bounding box techniques and/orother data optimization techniques are employed to discard the graphical data defining the top half of the object 284 before the coordinates of this graphical data are translated to screen relative by pipeline 58 and/or before other significantprocessing is performed on this data by pipeline 58.
In rendering the bottom half of the object 284, the pipeline 58 super-samples the data defining the bottom half of object 284 before storing this data in frame buffer 68. For illustrative purposes, assume that each pixel defining the bottom halfof object 284 is super-sampled by pipeline 58 into four pixels. Thus, if the super-sampled data stored in frame buffer 68 were somewhat directly rendered in region 249 without the processing performed by compositor 76, the image displayed by displaydevice 83 should appear to be magnified as shown in FIG. 12.
The compositor 76 is configured to blend the graphical data in frame buffers 66-69 and to composite or combine the blended data and the graphical data from frame buffer 65 such that the screen 247 displays the image shown by FIG. 10. Inparticular, the compositor 76 blends into a single pixel each set of four pixels that were previously super-sampled from the same pixel by pipeline 56. This blended pixel should have a color value that is a weighted average or a blend of the colorvalues of the four super-sampled pixels. Furthermore, the compositor 76 also blends into a single pixel each set of four pixels that were previously super-sampled from the sample pixel by pipeline 58. This blended pixel should have a color value thatis a weighted average or a blend of the color values of the four super-sampled pixels. Thus, the object 284 should appear in anti-aliased form within portions 266 and 268, as depicted in FIG. 10.
The super-sampling performed by pipelines 56-59 should improve the quality of the image displayed by display device 83. Furthermore, since each pipeline 56-59 is responsible for rendering only a portion of the image displayed by display device83, similar to the optimization mode, the speed at which a super-sampled image is rendered to display device 83 can be maximized.
Jitter Mode
Referring to FIG. 3, the operation and interaction of the client 52, pipelines 55-59, and the compositor 76 will now be described in more detail while each of the pipelines 55-59 is operating in the jitter mode. In the jitter mode, each pipeline56-59 is responsible for rendering the graphical data defining the entire 3D image to be displayed within region 249. Thus, each pipeline 56-59 refrains from discarding portions of the graphical data based on inputs received from slave controller 261,as described hereinabove for the optimization and super-sampling modes. Instead, each pipeline 56-59 renders the graphical data for each portion of the image visible within the entire region 249.
However, each pipeline 56-59 adds a small offset to the coordinates of each pixel rendered by the pipelines 56-59. The offset applied to the pixel coordinates is preferably different for each different pipeline 56-59. The different offsetsapplied by the different pipelines 56-59 can be randomly generated by each pipeline 56-59 and/or can be pre-programmed into each pipeline 56-59. After the pipelines 56-59 have applied the offsets to the pixel coordinates and have rendered to framebuffers 66-69, respectively, the compositor 76 combines the graphical representation defined by the data in each frame buffer 66-69 into a single representation that is rendered to the display device 83 for displaying. In combining the graphicalrepresentations, the compositor 76 averages or blends the color values at the same pixel locations in frame buffers 66-69 into a single color value for the same pixel location in the final graphical representation that is to be rendered to the displaydevice 83.
The aforementioned process of averaging multiple graphical representations of the same image should produce an image that has been jitter enhanced. The drawback to enhancing the image quality in this way is that each pipeline 56-59 renders theentire image to be displayed within region 249 instead of just a portion of such image as described in the optimization and super-sampling modes. Thus, the amount of time required to render the same image may be greater for the jitter mode as opposed tothe optimization and super-sampling modes. However, as compared to conventional systems 15 and 41, the amount of time required for the system 50 to render a jitter enhanced image should be significantly less than the amount of time required for eitherof the conventional systems 15 and 41 to produce the same jitter enhanced image.
In this regard, in performing jitter enhancing in a conventional system 15 or 41, a single pipeline 23 or 361439 usually renders the graphical data defining an image multiple times to enable jitter enhancement to occur. Each time the pipeline 23or 36-39 renders the graphical data, the pipeline 23 or 36-39 applies a different offset. However, in the present invention, a different offset is applied to the same graphical data via multiple pipelines 56-59. Therefore, to achieve the same level ofjitter enhancement of an image, it is not necessary for each pipeline 56-59 of system 50 to render the graphical data defining the image the same number of times as the single conventional pipeline 23 or 36-39. Thus, the system 50 should be able torender an jitter enhanced image faster than conventional systems 15 and 41.
To better illustrate the operation of the system 50 in the jitter mode, assume that the application 17 issues a command to display the 3D object 284 depicted in FIG. 10. In this example, graphical data defining the object is transmitted from theclient 52 to the master pipeline 55. The master pipeline 55 transmits this graphical data to each of the slave pipelines 56-59. Each of the slave pipelines 56-59 renders the graphical data defining the 3D object 284 to frame buffers 66-69,respectively. In rendering the graphical data, each pipeline 56-59 adds a small offset to each set of coordinate values within the graphical data defining the object 284. The offset added by each pipeline 56-59 is preferably different and small enoughsuch that the graphical representations of the object, as defined by frame buffers 66-69, would substantially but not exactly overlay one another, if each of these representations were displayed by the same display device 83.
As an example, pipeline 56 may add the value of 0.1 to each coordinate rendered by the pipeline 56, and pipeline 57 may add the value of 0.2 to each coordinate rendered by the pipeline 56. Further, pipeline 58 may add the value of 0 to eachcoordinate rendered by the pipeline 58, and the pipeline 59 may add the value of -0.2 to each coordinate rendered by the pipeline 59. Note that it is not necessary for the same offset to be added to each coordinate rendered by a particular pipeline56-59. For example, one of the pipelines 56-59 could be configured to add the value of 0.1 to each x-coordinate value rendered by the one pipeline 56-59 and to add the value of 0.2 to each y-coordinate value and z-coordinate value rendered by the onepipeline 56-59.
The graphical data in frame buffers 66-69 is transmitted to compositor 76, which forms a single graphical representation of the object 284 based on each of the graphical representations from frame buffers 66-69. In this regard, the compositor 76averages or blends into a single color value the color values of each pixel from frame buffers 66-69 having the same screen relative coordinate values. Each color values calculated by the compositor 76 is then assigned to the pixel having the samecoordinate values as the pixels that were averaged or blended to form the color value calculated by the compositor 76.
As an example, assume that color values stored in frame buffers 66-69 for the pixel having the coordinate values (1000, 1000, 0) are a, b, c, and d, respectively, in which a, b, c, and d represent four different numerical values. In thisexample, the compositor 76 may calculate a new color value, n, based on the following equation: n=(a+b+C+d)/4. This new color value, n, is then transmitted to display device 83 as the color value for the pixel having coordinates (1000, 1000, 0). Notethat a different algorithm may be used to calculate the new color value and that different weightings may be applied to the values being averaged.
By performing the above-described process for each pixel represented in frame buffers 66-69, the compositor 76 produces graphical data defining a jitter enhanced image of the 3D object 284. This data is rendered to the display device 83 todisplay the jitter enhanced image of the object 284.
It should be noted that it is not necessary for each of the pipelines 56-59 to operate in only one mode of operation. For example, it is possible for the pipelines 56-59 to operate in both the optimization mode and the jitter mode. As anexample, the region 249 could be divided into two portions according to the techniques described herein for the optimization mode. The pipelines 56 and 57 could be responsible for rendering graphical data within one portion of the region 249, andpipelines 58 and 59 could be responsible for rendering within the remaining portion of the region 249. Furthermore, the pipelines 56 and 57 could render jitter enhanced and/or anti-aliased images within their portion of region, and pipelines 58 and 59could render jitter enhanced and/or anti-aliased images within the remaining portion of region 249. The modes of pipelines 56-59 may be mixed according to other combinations in other embodiments.
Furthermore, it is not necessary for the application 17 to be aware of which mode or combination of modes are being implemented by pipelines 55-59, since the operation of the application 17 is the same regardless of the implemented mode orcombination of modes. In other words, the selection of the mode or modes implemented by the pipelines 55-59 can be transparent to the application 17.
It should be noted that there are a variety of methodologies that may be employed to enable the selection of the mode or modes performed by the system 50. In the preferred embodiment, a user is able to provide inputs via input device 115 ofclient 52 (FIG. 4) indicating which mode or modes the user would like the system 50 to implement. The client 52 is designed to transmit the user's mode input to master pipeline 55 over LAN 62. The slave controller 261 of the master pipeline 55 (FIG. 5)is designed to then provide appropriate input to each slave pipeline 56-59 instructing each slave pipeline 56-59 which mode to implement based on the mode input received from client 52. The slave controller 261 also transmits control information tocompositor 76 via connection 331 (FIG. 3) indicating which mode is being implemented by each pipeline 56-59. The compositor 76 then utilizes this control information to appropriately process the graphical data from frame buffers 76, as further describedherein. There are various other methodologies and configurations that may be employed to provide the slave pipelines 56-59 and/or compositor 76 with the necessary mode information for enabling the pipelines 56-59 and compositor 76 to operate as desired. For example, the control information may be included in the data transmitted from the master pipeline 55 to the slave pipelines 56-59 and then from the slave pipelines 56-59 to the compositor 76.
It should be noted that master pipeline 55 has been described herein as only rendering 2D graphical data. However, it is possible for master pipeline 55 to be configured to render other types of data, such as 3D image data, as well. In thisregard, the master pipeline 55 may also include an OGL daemon, similar to the OGL daemon 205 within the slave pipelines 56-59. The purpose for having the master pipeline 55 to only execute graphical commands that do not include 3D image data is toreduce the processing burden on the master pipeline 55, since the master pipeline 55 performs various functionality note performed by the slave pipelines 56-59. In this regard, executing graphical commands including only 2D image data is generally lessburdensome than executing commands including 3D image data. However, it may be possible and desirable in some implementations to allow the master pipeline 55 to share in the execution of graphical commands that include 3D image data. Furthermore, itmay also be possible and desirable in some implementations to allow the slave pipelines 56-69 to share in the execution of graphical commands that do not include 3D image data (e.g., commands that only include 2D graphical data).
In addition, a separate computer system may be used to provide the functionality of controlling the graphics pipelines. For example, FIG. 13 depicts another embodiment of the graphical acceleration unit 95. This embodiment includes multiplepipelines 315-319 configured to render data similar to pipelines 55-59, respectively. However, a separate computer system, referred to as master server 322, is employed to route graphical data received from client 52 to pipelines 315-319 and to controlthe operation of pipelines 315-319, similar to how slave control 261 of FIG. 5 controls the operation of pipelines 56-59. Other configurations may be employed without departing from the principles of the present invention. Furthermore, as previouslyset forth, it is not necessary to implement each pipeline 55-59 and the client 52 via a separate computer system. A single computer system may be used to implement multiple 55-59 and/or may be used to implement the client 52 and at least one pipeline55-59.
It should be further noted that the present invention has been described as utilizing X Protocol and OpenGL Protocol to render graphical data. However, other types of protocols may be utilized without departing from the principles of the presentinvention.
Single Logical Screen Implementation
The graphical acceleration unit 95 described herein may be utilized to implement a single logical screen (SLS) graphical system, similar to the conventional system 41 shown in FIG. 2. As an example, refer to FIG. 14, which depicts an SLSgraphical display system 350 in accordance with the present invention. The system 350 includes a client 52 storing the graphical application 17 that produces graphical data to be rendered, as described hereinabove. Any graphical command produced by theapplication 17 is preferably transmitted to SLS server 356, which may be configured similarly to the conventional SLS server 45 of FIG. 2. More specifically, the SLS server 356 is configured to interface each command received from the client 52 withmultiple graphical acceleration units 95a-95d similar to how conventional SLS server 45 interfaces commands received from client 42 with each graphics pipeline 36-39. The SLS server 356 may be implemented in hardware, software, or a combination thereof,and in the preferred embodiment, the SLS server 356 is implemented as a stand-alone computer workstation or is implemented via a computer workstation that is used to implement the client 52. However, there are various other configurations that may beused to implement the SLS server 356 without departing from the principles of the present invention.
Each of the graphical acceleration units 95a-95d, according to the techniques described herein, is configured to render the graphical data received from SLS server 356 to a respective one of the display devices 83a-83d. Note that theconfiguration of each graphical acceleration unit 95a-95d may be identical to the graphical acceleration unit 95 depicted by FIG. 3 or FIG. 13, and the configuration of each display device 83a-83d may be identical to the display device 83 depicted inFIGS. 3 and 13. Moreover, an image defined by the graphical data transmitted from the application 17 may be partitioned among the display devices 83a-83d such that the display devices 83a-83d collectively display a single logical screen similar to howdisplay devices 31-34 of FIG. 2 display a single logical screen.
To better illustrate the operation of the system 350, assume that a user would like to display an image of the 3D object 284 (FIG. 10) via the display devices 83a-83d as a single logical screen. FIG. 15 depicts how the object 284 may bedisplayed by display devices 83a-83d in such an example. More specifically, in FIG. 15, the display device 83a displays the top half of the object 284, and the display device 83c displays the bottom half of the object 284.
In the foregoing example, the client 52 transmits a command for displaying the object 284. The command includes the graphical data defining the object 284 and is transmitted to SLS server 356. The SLS server 356 interfaces the command with eachof the graphical acceleration units 95a-95d. Since the object 284 is not to be displayed by display devices 83b and 83d, the graphical acceleration units 95b and 95d fail to render the graphical data from the command to display devices 83b and 83d. However, graphical acceleration unit 95a renders the graphical data defining the top half of the object 284 to display device 83a, and graphical acceleration unit 95c renders the graphical data defining the bottom half of the object 284 to display device83c. In response, the display device 83a displays the top half of the object 284, and the display device 83c displays the bottom half of the object 284, as shown by FIG. 15.
Note that the graphical acceleration units 95a and 95c may render their respective data based on any of the modes of operation previously described. For example, the master pipeline 55 (FIG. 3) of the graphical acceleration unit 95a preferablyreceives the command for rendering the object 284 and interfaces the graphical data from the command to slave pipelines 56-59 (FIG. 3) of the graphical acceleration unit 95a. These pipelines 56-59 may operate in the optimization mode, the super-samplingmode, and/or the jitter mode, as previously described hereinabove, in rendering the graphical data defining the top half of the object 284.
In addition, the master pipeline 55 (FIG. 3) of the graphical acceleration unit 95c preferably receives the command for rendering the object 284 and interfaces the graphical data from the command to slave pipelines 56-59 (FIG. 3) of the graphicalacceleration unit 95c. These pipelines 56-59 may operate in the optimization mode, the super-sampling mode, and/or the jitter mode, as previously described hereinabove, in rendering the graphical data defining the bottom half of the object 284.
Note that the master pipeline 55 (FIG. 3) of each graphical acceleration unit 95a-95d may employ bounding box techniques to optimize the operation of the system 350. In particular, the master pipeline 55 (FIG. 3) may analyze bounding box data aspreviously described hereinabove to determine quickly whether the graphical data associated with a received command is to be rendered to the display device 83a-83d that is coupled to the unit 95a-95d. If the graphical data of the received command is notto be rendered to the display device 83a-83d coupled to the graphical acceleration unit 95a-95d, then the master server 55 of the graphical acceleration unit 95a-95d may be configured to discard the command before transmitting the graphical data of thecommand to any of the slave pipelines 56-59 and/or before performing any significant processing of the command. However, if any of the graphical data of the received command is to be rendered to the display device 83a-83d coupled to the graphicalacceleration unit 95a-95d, then the unit 95a-95d can be configured to further process the command as described herein.
It should be noted that the system 350 can be scaled as needed in order to achieve a desired level of processing speed and/or image quality. In this regard, the number of graphical acceleration units 95a-95d and associated display devices83a-83d can be increased or decreased as desired depending on how large or small of a single logical screen is desired. Further, the number of slave pipelines 56-59 (FIG. 3) within each graphical acceleration unit 95a-95d can be increased or decreasedbased on how much processing speed and/or image quality is desired for each display device 83a-83d. Note that the number of slave pipelines 56-59 within each unit 95a-95d does not have to be the same, and the modes and/or the combinations of modesimplemented by each unit 95a-95d may be different.
Furthermore, in the embodiment shown by FIG. 3, mode inputs from the user were provided to the master pipeline 55, which controlled the mode of operation of the slave pipelines 55-59 and the compositor 76. In the embodiment shown by FIG. 14,such inputs may be similarly provided to the master pipeline 55 within each graphical acceleration unit 95a-95d via the client 52 and the SLS server 356. However, as previously set forth hereinabove, there are various other methodologies that may beemployed to control the mode of operation of the pipelines 56-59 and the compositor 76.
The Compositor
As mentioned briefly hereinbefore, the compositor may be employed by a computer graphical display system of the present invention. In this regard, computer graphical display system 50 (depicted in FIG. 16, for example) includes a client 52, amaster graphical pipeline 55, and one or more slave graphical pipelines 55-59. The master pipeline 55 receives graphical data from an application 17 stored in the client 52. The master pipeline 55 preferably renders two dimensional (2D) graphical datato frame buffer 65 and routes three dimensional (3D) graphical data to slave pipelines 56-59, which render the 3D graphical data to frame buffers 66-69, respectively. The frame buffers 65-69 each output a stream of graphical data to the compositor 76,which is configured to composite or combine each of the data streams into a single, composite data stream. The composite data stream then may be provided to the display device 83, for example, for displaying an image thereon. A preferred embodiment ofthe compositor of the present invention will now be described in greater detail. Note that implementations for the compositor, other than the ones expressly described herein, may be employed to implement the graphical display system of the presentinvention.
Embodiments of the compositor and associated methodology of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In a preferred embodiment, compositor 76 includes an input mechanism 391, an outputmechanism 392, and a controller 393. As described in detail hereinafter, controller 393 enables input mechanisms 391 to approximately combine or composite the data streams from the various pipelines so as to provide a composite data stream which issuitable for rendering. In order to facilitate control of input mechanism 391, compositor 76 may receive control information from client 52, with such control information being provided to the controller 392 via a transmission media 394, such as a USB,for example, or one of the pipelines. FIG. 3 depicts an embodiment where the control information is provided to the compositor 76 over connection 331 from the master pipeline 55.
As embodiments of the compositor, components thereof, and associated functionality may be implemented in hardware, software, firmware, or a combination thereof, those embodiments implemented at least partially in software can be adaptable to runon different platforms and operating systems. In particular, logical functions implemented by the compositor may be provided as an ordered listing of executable instructions that can be embodied in any computer-readable medium for use by or inconnection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device, and executethe instructions.
In the context of this document, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. Thecomputer readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electro-magnetic, infrared, or semi-conductor system, apparatus, device, or propagation medium. More specific examples (a non-exhaustive list) of thecomputer-readable medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable, programmable, read-only memory (EPROM or Flashmemory), an optical fiber, and a portable compact disk read-only memory (CDROM). Note that the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, viafor instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
Reference will now be made to the flowcharts of FIGS. 17 and 18, which depict functionality of preferred embodiments of the compositor. In this regard, each block of the flowcharts represents one or more executable instructions for implementingthe specified logical function or functions. It should be noted that in some alternative implementations, the functions noted in the various blocks may occur out of the order depicted in the respective figures. For example, two blocks shown insuccession in FIG. 18, may, in fact, be executed substantially concurrently where the blocks may sometimes be executed in the reverse order depending upon the functionality involved.
Referring now to FIG. 17, the functionality of an embodiment of the compositor may be construed as beginning at block 402, where 2D and 3D graphical data relating to an image to be rendered, such as graphical data provided from multipleprocessing pipelines, for instance, are received. In block 404, the graphical data are combined to form a composite data stream containing data corresponding to the image. Thereafter, the compositor provides the composite data stream (block 406), whichmay be utilized by a single display device for displaying the image.
In regard to the functionality or process depicted in FIG. 18, that process may be construed as beginning at block 410 where information corresponding to a particular compositing mode or format is received. Thereafter, such as depicted in blocks412, 414 and 416, determinations are made as to whether the compositing mode information corresponds to one of an optimization mode (block 412), a jitter mode (block 414), or a super-sample mode (block 416).
If it is determined that the information corresponds to the optimization mode, the process may proceed to block 418 where information corresponding to the allocation of pipeline data is received. More specifically, in this mode, each graphicalprocessing pipeline is responsible for processing information relating only to a portion of the entire screen resolution being processed. Therefore, the information corresponding to the allocation of pipeline data relates to which portion of the screencorresponds to which pipeline. Proceeding to block 420, data is received from each pipeline with the data from each pipeline corresponding to a particular screen portion. (It should be noted that the pipeline which processes the 2D graphicalinformation may process such 2D graphical data for the entire screen resolution; thus, the description of blocks 418 and 420 relate most accurately to the processing of 3D graphical data). Thereafter, such as in block 422, compositing of pipeline datawith regard to the aforementioned allocation of data is enabled. In block 424, a composite data stream, e.g., a data stream containing pixel data corresponding to the entire screen resolution (2000 pixels by 2000 pixels, for example) is provided.
If it is determined in block 414 that the information received in block 410 corresponds to the jitter or accumulate mode, the process may proceed to block 426 where pixel data from each pipeline corresponding the entire screen resolution, e.g.,2000 pixels by 2000 pixels, is received. Thereafter, such as in block 428, an average value for each pixel may be determined utilizing the pixel data from each of the pipelines. After block 428, the process may proceed to block 424, as describedhereinbefore.
If it is determined in block 416 that the information received in block 410 corresponds to the super-sample mode, the process may proceed to block 430. As depicted therein, information corresponding to the allocation of pipeline data isreceived. For instance, the 3D graphical data may be equally divided among the pipelines designated for processing 3D data. Continuing with this representative example, each of the pipelines also may be allocated a screen portion corresponding to 1000pixels by 1000 pixels. Thereafter, such as depicted in block 432, data is received from each pipeline that corresponds to the aforementioned screen portion allocation; however, the data of each pipeline has been super-sampled during processing so thatthe received data from each pipeline corresponds to a screen size that is larger than its screen portion allocation. For example, data from each pipeline may correspond to a screen resolution of 2000 pixels by 2000 pixels, e.g., each of the horizontaland vertical dimensions may be doubled. Thus, each pipeline provides four pixels of data for each pixel to be rendered. In other configurations, each of the pipelines may provide various other numbers of pixels of data for each pixel to be rendered.
Proceeding to block 434, the super-sampled data is then utilized to determine an average value for each pixel to be rendered by each pipeline. More specifically, since each pixel to be rendered previously was super-sampled into four pixels,determining an average value for each pixel preferably includes down-sampling each grouping of four pixels back into one pixel. Thus, in the aforementioned example, data from each pipeline is down-sampled and the data from each pipeline, which isrepresentative of a portion of the entire screen resolution, is then composited in block 424, as described hereinbefore.
After the composite data stream has been provided, such as depicted in block 424, a determination may then be made as to whether stereo output is desired (block 436). If it is determined that stereo processing is desired, the process may proceedto block 438 where stereo processing is facilitated (described in detail hereinafter). If is was determined in block 436 that stereo processing was not desired or, alternatively, after facilitating stereo processing in block 438, the process may proceedto block 440. As depicted in block 440, a determination may be made as to whether a digital video output is desired. If a digital video output is desired, the process may proceed to block 442 for appropriate processing. Alternatively, if an analogoutput is desired, the process may proceed to block 444 where the composite data stream may be converted to an analog data stream.
Referring now to FIG. 19, which depicts a preferred embodiment of input mechanism 391 and output mechanism 392, input 391 mechanism is configured to receive multiple data streams, e.g., data streams 455-459. In particular, the data streams areprovided by pipelines, such as pipelines 55-59 of FIG. 11, with the data being intermediately provided to corresponding frame buffer, such as buffers 65-69. Each of the data streams 455-459 is provided to a buffer assembly of the input mechanism 391that preferably includes two or more buffers, such as frame buffers or line buffers, for example. More specifically, in the embodiment depicted in FIG. 19, data stream 455 is provided to buffer assembly 460, which includes buffers 461 and 462, datastream 456 is provided to buffer assembly 464, which includes buffers 465 and 466, data stream 457 is provided to buffer assembly 468, which includes buffers 469 and 470, data stream 458 is provided to buffer assembly 472, which includes buffers 473 and474, and data stream 459 is provided to buffer assembly 476, which includes buffers 477 and 478. Although data stream 459 is depicted as comprising 2D data, data which may be provided by the master pipeline, for instance, the 2D data may be provided toany of the frame buffer assemblies.
The buffers of each buffer assembly cooperates so that a continuous output stream of data may be provided from each of the buffer assemblies. More specifically, while data from a particular data stream is being written to one of the pair ofbuffers of a buffer assembly, data is being read from the other of the pair. In other embodiments, buffer assemblies may be provided with more than two buffers that are adapted to provide a suitable output stream of data. Additionally, in still otherembodiments, the pipelines may provide pixel data directly to respective compositing elements without intervening buffers being provided therebetween.
In the embodiment depicted in FIG. 19, each of the frame buffer assemblies communicates with a compositing element. For example, buffer assembly 460 communicates with compositing element 480, buffer assembly 464 communicates with compositingelement 481, buffer assembly 468 communicates with compositing element 482, buffer assembly 472 communicates with compositing elements 483, and buffer assembly 476 communicates with compositing element 484. So configured, each buffer assembly is able toprovide its respective compositing element with an output data stream.
Each compositing element communicates with an additional compositing element for forming the composite data stream. More specifically, compositing element 480 communicates with compositing element 481, compositing element 481 communicates withcompositing element 482, compositing element 482 communicates with compositing elements 483, and compositing element 483 communicates with compositing element 484. So configured, data contained in data stream 455 is presented to compositing element 480via buffer assembly 460. In response thereto, compositing element 480 outputs data in the form of data stream 490, which is provided as an input to compositing element 481. Compositing element 481 also receives an input corresponding to data containedin data stream 456, via buffer assembly 464. Compositing element 481 then combines or composites the data provided from buffer assembly 464 and compositing element 480 and outputs a data stream 491. Thus, data stream 491 includes data corresponding todata streams 455 and 456. Compositing element 482 receives data stream 491 as well as data contained within data stream 457, which is provided to compositing element 482 via buffer assembly 468. Compositing element 482 composites the data from datastream 491 and data stream 457, and then outputs the combined data via data stream 492. Compositing element 483 receives data contained in data stream 492 as well as data contained within data stream 458, which is provided to compositing element 483 viaframe buffer 472. Compositing element 483 composites the data from data stream 492 and data stream 458, and provides an output in the form of data stream 493. Data stream 493 is provided as an input to compositing element 484. Additionally,compositing element 484 receives data corresponding to data stream 459, which is provided via buffer assembly 476. Compositing element 484 then composites the data from data stream 459 and data stream 459, and provides a combined data stream output ascomposite data stream 494. Composite data stream 494 then is provided to output mechanism 392.
Compositing of the multiple data stream preferably is facilitated by designating portions of a data stream to correspond with particular pixel data provided by the aforementioned pipelines. In this regard, compositing element 480, which is thefirst compositing element to provide a compositing data stream, is configured to generate a complete frame of pixel data, i.e., pixel data corresponding to the entire resolution to be rendered. This complete frame of pixel data is provided bycompositing element 480 as a compositing data stream. In response to receiving the compositing data stream, each subsequent compositing element may then add pixel data, i.e., pixel data corresponding to its respective pipeline, to the compositing datastream. After each compositing element has added pixel data to the compositing data stream, the data stream then contains pixel data corresponding to data from all of the aforementioned pipelines. Such a data stream, i.e., a data stream containingpixel data corresponding to data from all of the processing pipelines, may be referred to herein as a combined or composite data stream.
The first compositing element to provide pixel data to a compositing data stream, e.g., compositing element 480, also may provide video timing generator (VTG) functionality. Such VTG functionality may include, for example, establishinghorizontal scan frequency, establishing vertical scan frequency, and establishing dot clock, among others.
Generation of a composite data stream will now be described with reference to the schematic diagrams of FIGS. 10 and 19. As mentioned briefly hereinbefore in regard to FIG. 10, a particular slave pipeline is responsible for rendering graphicaldata displayed in each of screen portions 266-269. Additionally, 2D graphical information corresponding to the entire screen resolution, e.g., screen 247, is processed by a separate pipeline. For the purpose of the following discussion, graphical dataassociated with screen portion 266 corresponds to data stream 455 of FIG. 19, with screen portions 267, 268 and 269 corresponding to data streams 456, 457 and 458 respectively. Additionally, 2D graphical data, which is represented by window 245 of FIG.10, corresponds to data stream 459 of FIG. 19.
As described hereinbefore, data streams 455-459 are provided to their respective buffer assemblies where data is written to one of the buffers of each of the respective buffer assemblies as data is read from the other buffer of each of theassemblies. The data then is provided to respective compositing elements for processing. More specifically, receipt of data by compositing element 480 initiates generation of an entire frame of data by that compositing element. Thus, in regard to therepresentative example depicted in FIG. 10, compositing element 480 generates a data frame by 2000 pixels by 2000 pixels, e.g., a data corresponding to the entire screen resolution 247 of FIG. 10. Compositing element 480 also is programmed to recognizethat data provided to it corresponds to pixel data associated with a particular screen portion, e.g., screen portion 266. Therefore, when constructing the frame of data corresponding to the entire screen resolution, compositing element 480 utilizes thedata provided to its, such as via its buffer assembly, and appropriately inserts that data into the frame of data. Thus, compositing element 480 inserts pixel data corresponding to screen portion 266, i.e., pixels (700, 1300) to (1000, 1000), into theframe. Those pixels not corresponding to screen portion 266 may be represented by various other pixel information, as desired. For instance, in some embodiments, the data corresponding to remaining portions of the frame may be left as zero, forexample. Thereafter, the generated frame of data, which now includes pixel data corresponding to screen portion 266, may be provided from compositing element 480 at compositing data stream 490. Compositing data stream 490 then is provided to a nextcompositing element for further processing.
As depicted in FIG. 19, compositing data stream 490 is received by compositing element 481. Compositing element 481 also is configured to receive data from data stream 456, such as via buffer assembly 464, that may contain data corresponding toscreen portion 267 of FIG. 10, for example, Thus, compositing element 481 may receive data corresponding to pixels (1000, 1300) to (1300, 1000). Compositing element 481 is configured to insert the pixel data corresponding to pixels of screen portion 267into the compositing data stream by replacing any data of the stream previously associated with, in this case, pixels (1000, 1300) to (1300, 1000), with data contained in data stream 456. Thereafter, compositing element 481 is able to provide acompositing data stream 491, which contains pixel data corresponding to the entire screen resolution as well as processed pixel data corresponding to pixels (700, 1300) to (1300, 1000), i.e., screen portions 266 and 267.
Compositing data stream 491 is provided to the next compositing element, e.g., composite element 482. Additionally, compositing element 482 receives pixel data from data stream 457, such as via buffer assembly 468, that corresponds to screenportion 268. Compositing element 482 inserts pixel data from data stream 457 into the compositing data stream and provides a compositing data stream 492 containing data corresponding to the entire frame resolution as well as processed pixel datacorresponding to screen portions 266, 267 and 268. Compositing data stream 492 then is provided to compositing element | | | |
