Display apparatus for performing voice control and voice controlling method thereof

More than one reissue application has been filed for the reissue of U.S. Pat. No. 9,711,149. The reissue applications are the present application and U.S. application Ser. No. 16/515,466. U.S. application Ser. No. 16/515,466, filed on Jul. 18, 2019, is a reissue of U.S. Pat. No. 9,711,149, which was filed as U.S. application Ser. No. 14/515,781 on Oct. 16, 2014 and issued on Jul. 18, 2017, the disclosures of which are incorporated herein by reference in their entirety. The present application is a continuation reissue of U.S. application Ser. No. 16/515,466 filed on Jul. 18, 2019 and a reissue of U.S. Pat. No. 9,711,149.

This application claims priority from Korean Patent Application No. 10-2014-0009388, filed on Jan. 27, 2014, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.

1. Field

Methods and devices of manufacture consistent with exemplary embodiments relate to a display apparatus and a voice controlling method thereof, and more particularly, to a display apparatus for determining a voice input of a user to perform an operation and a voice controlling method thereof.

2. Description of the Related Art

As display apparatuses have been gradually becoming more multifunctional and advanced, various input methods for controlling the display apparatuses have been developed. For example, an input method using a voice control technology, an input method using a mouse, an input method using a touch pad, an input method using a motion sensing remote controller, etc. have been developed.

However, there are several kinds of disadvantages in using voice control technology. For example, if a voice uttered by a user is a simple keyword having no verb, a different operation from that intended by the user may be performed.

In other words, if the display apparatus misrecognizes the voice uttered by the user, the display apparatus may not be controlled as the user wants.

Exemplary embodiments address at least the above disadvantages and other disadvantages not described above. Also, the exemplary embodiments are not required to overcome the disadvantages described above, and an exemplary embodiment may not overcome any of the disadvantages described above.

One or more exemplary embodiments provide a display apparatus for determining a voice input of a user to perform an operation corresponding to an intention of the user and a method of controlling a voice.

According to an aspect of an exemplary embodiment, there is provided a voice controlling method of a display apparatus, the method including: receiving a voice of a user; converting the voice into text; sequentially changing and applying a plurality of different determination criteria to the text until a control operation corresponding to the text is determined; and performing the determined control operation to control the display apparatus.

The sequentially changing and applying the plurality of different determination criteria may include determining whether the text corresponds to a title of an object displayed on a screen of the display apparatus; and in response to determining that the text corresponds to the title of the object, determining an operation corresponding to the object as the control operation.

The determining whether the text corresponds to the title of the object may include, in response to a part of the title of the object being displayed and the text corresponding to at least a portion of the displayed part of the object, determining that the text corresponds to the title of the object.

The determining whether the text corresponds to the title of the object may include, in response to only a part of one word included in the title of the object being displayed and the text corresponding to the whole one word, determining that the text corresponds to the title of the object.

The object may include at least one of a content title, an image title, a text icon, a menu name, and a number that are displayed on the screen.

The sequentially changing and applying the plurality of different determination criteria may further include: in response to determining that the text does not correspond to the title of the object, determining whether the text corresponds to a stored command; and in response to determining that the text corresponds to the stored command, determining an operation corresponding to the stored command as the control operation.

The stored command may include at least one of a command for controlling power of the display apparatus, a command for controlling a channel of the display apparatus, and a command for controlling a volume of the display apparatus.

The sequentially changing and applying the plurality of different determination criteria may include: in response to determining that the text does not correspond to the stored command, determining whether a meaning of the text is analyzable; and in response to determining that the meaning of the text is analyzable, analyzing the meaning of the text and determining an operation of displaying a response message corresponding to the analysis result as the control command.

The sequentially changing and applying the plurality of different determination criteria may include in response to determining that the meaning of the text is not analyzable, determining an operation of a search using the text as a keyword, as the control operation.

According to an aspect of another exemplary embodiment, there is provided a display apparatus including: a voice input circuit configured to receive a voice of a user; a voice converter configured to convert the voice into text; a storage configured to store a plurality of determination criteria that are different from one another; and a controller configured to sequentially change and apply a plurality of different determination criteria to the text until a control operation corresponding to the text is determined, and perform the determined control operation.

The controller may be configured to sequentially change and apply the plurality of different determination criteria to the text by determining whether the text corresponds to a title of an object displayed on a screen of the display apparatus and, in response to determining that the text corresponds to the title of the object, determining an operation corresponding to the object as the control operation.

The controller may be configured to, in response to only a part of the title of the object being displayed and determining that the text corresponds to the part of the title of the object being displayed, determine that the text corresponds to the title of the object.

The controller may be configured to, in response to only a part of one word included in the title of the object being displayed and determining that the text corresponds to the whole one word, determine that the text corresponds to the title of the object.

The object may include at least one of a content title, an image title, a text icon, a menu name, and a number that are displayed on the screen.

The controller may be configured to sequentially change and apply the plurality of different determination criteria to the text by, in response to determining that the text does not correspond to the title of the object, determining whether the text corresponds to a stored command, and in response to determining that the text corresponds to the stored command, determining an operation corresponding to the stored command as the control operation.

The stored command may include at least one of a command for controlling power of the display apparatus, a command for controlling a channel of the display apparatus, and a command for controlling a volume of the display apparatus.

The controller may be configured to sequentially change and apply the plurality of different determination criteria to the text by, in response to determining that the text does not correspond to the stored command, determining whether a meaning of the text is analyzable, and, in response to determining that the meaning of the text is analyzable, analyzing the meaning of the text and determines an operation of displaying a response message corresponding to the analysis result, as the control operation.

The controller may be configured to sequentially change and apply the plurality of different determination criteria to the text by, in response to determining that the meaning of the text is not analyzable, determining an operation of a search using the text as a keyword, as the control operation.

The plurality of different determination criteria may include criteria of whether the text corresponds to a title of a displayed object, whether the text corresponds to a stored command, whether the text is grammatically analyzable, and whether the text refers to a keyword.

According to an aspect of another exemplary embodiment, there may be provided a voice controlling method of a display apparatus, the voice controlling method including receiving a voice of a user; converting the voice into text; applying at least two tiers of hierarchical criteria the text to determine a control operation corresponding to the text; and controlling the display apparatus according to the determined control operation.

Exemplary embodiments are described in greater detail with reference to the accompanying drawings.

In the following description, the same drawing reference numerals are used for the same elements even in different drawings. The matters defined in the description, such as detailed construction and elements, are provided to assist in a comprehensive understanding of the exemplary embodiments. Thus, it is apparent that the exemplary embodiments can be carried out without those specifically defined matters. Also, well-known functions or constructions are not described in detail since they would obscure the exemplary embodiments with unnecessary detail.

is a block diagram illustrating a structure of a display apparatus according to an exemplary embodiment. Referring to, a display apparatusincludes a voice input circuit, a voice converter, a controller, and a storage.

The display apparatusmay receive a voice of a user through the voice input circuitand convert the voice into text using the voice converter. Here, the display apparatusmay sequentially change a plurality of different determination criteria until a control operation corresponding to the converted text is determined and then determine the control operation corresponding to the converted text.

The display apparatusmay be a display apparatus such as a smart TV, but this is only an . Alternatively, the display apparatusmay be realized as, for example, a desktop personal computer (PC), a tablet PC, a smartphone, or the like or may be realized as another type of input device such as a voice input device.

The voice input circuitis an element that receives the voice of the user. In detail, the voice input circuitmay include a microphone and associated circuitry to directly receive the user's voice as sound and convert the sounds to an electric signal, or may include circuitry to receive an electric signal corresponding to the user's voice input to the display apparatusthrough a microphone that is connected to the display apparatusby wire or wireless. The voice input circuittransmits the signal corresponding to the user's voice to the voice converter.

The voice converterparses a waveform of a characteristic of the user's voice signal (i.e., a characteristic vector of the user's voice signal) to recognize words or a word string corresponding to the voice uttered by a user and outputs the recognized words as text information.

In detail, the voice convertermay recognize the words or the word string uttered by the user from the user voice signal by using at least one of various recognition algorithms such as a dynamic time warping method, a Hidden Markov model, a neural network, etc. and convert the recognized voice into text. For example, if the Hidden Markov model is used, the voice converterrespectively models a time change and a spectrum change of the user voice signal to detect a similar word from a stored language database (DB). Therefore, the voice convertermay output the detected word or words as text.

The voice input circuitand the voice converterhave been described as elements that are installed in the display apparatusin the present exemplary embodiment, but this is only an example. Alternatively, the voice input circuitand the voice convertermay be realized as external devices.

The controllerperforms a control operation corresponding to the user voice input through the voice input circuit. The controllermay start a voice input mode according to a selection of the user. If the voice input mode starts, the controllermay activate the voice input circuitand the voice converterto receive the user's voice. If the user voice is input when a voice input mode is active, the controlleranalyzes an intention of the user by using a plurality of different determination criteria stored in the storage. The controllerdetermines a control operation according to the analysis result to perform an operation of the display apparatus

In detail, the controllerdetermines whether the converted text corresponds to a title of an object in a displayed screen. If the converted text corresponds to the title of the object, the controllerperforms an operation corresponding to the object. For example, the controllermay perform an operation matching the object. In detail, the object may include at least one of a content title, an image title, a text icon, a menu name, and a number displayed on the screen.

According to an exemplary embodiment, if only a part of the title of the object is displayed, and only a part of the converted text matches at least a part of the title of the displayed object, the controllerdetermines that the converted text corresponds to the title of the object. For example, if only “Stairway” is displayed of a content title “Stairway to Heaven,” and the converted text “stair” is input, the controllermay determine that the text corresponds to the title.

According to another exemplary embodiment, if only a part of one word included in the title of the object is displayed, and the converted text matches the whole one word, the controllerdetermines that the converted text corresponds to the title of the object. For example, if only “Stair” is displayed of the content title “Stairway to Heaven,” and the converted text “stairway” is input, the controllermay determine the converted text corresponds to the title.

If the controllerdetermines the converted text does not correspond to the title of the object, the controllerdetermines whether the converted text corresponds to a command stored in the storage. If the controllerdetermines the converted text corresponds to the command stored in the storage, the controllerperforms an operation corresponding to the command. Alternatively, the controllermay perform an operation that matches the command.

Also, if the controllerdetermines that the converted text does not correspond to the command stored in the storage, the controllerdetermines whether a meaning of the converted text is analyzable. If the controllerdetermines that the meaning of the converted text is analyzable, the controllermay analyze the meaning of the converted text and display a response message corresponding to the analysis result.

If the controllerdetermines the meaning of the converted text is not analyzable, the controllermay perform a search by using the converted text as a keyword.

As described above, the controllermay directly perform the work of analyzing the user voice signal and converting the user voice signal into converted text. However, according to other exemplary embodiments, the controllermay transmit the user voice signal to an external server apparatus, and the external server apparatus may convert the user voice signal into text. Also, the controllermay be provided with the converted text. The external server apparatus that converts a user voice signal into text may be referred to as a voice recognition apparatus for convenience of description. An operation of the display apparatusthat operates along with a voice recognition apparatus to convert a voice into a text will be described in detail in a subsequent exemplary embodiment.

The storageis an element that stores various types of modules for driving the display apparatus. The storagemay store a plurality of determination criteria and a plurality of commands for providing a voice recognition effect. For example, the storagemay store software including a voice conversion module, a text analysis module, a plurality of determination criteria, a control analysis criteria, a base module, a sensing module, a communication module, a presentation module, a web browser module, and a service module. The plurality of determination criteria may include whether converted text corresponds to a title of an object displayed on the screen, whether the converted text corresponds to a stored command, whether the converted text is grammatically analyzable, and whether the converted corresponds to a search keyword. The controllermay sequentially move through the plurality of determination criteria and apply the plurality of determination criteria sequentially in order to the converted text until a control operation is determined. In other words, the determination criteria represent a hierarchy of different tiers. That is, first it is determined whether the converted text corresponds to a displayed object in a first tier, and then if not, it is determined whether the converted text corresponds to a stored command in a second tier, and so on. After the control operation is determined, the control operation is performed.

According to an exemplary embodiment, the storagemay store at least one of a command for controlling power of the display apparatus, a command for controlling a channel of the display apparatus, and a command for controlling a volume of the display apparatus. A command may be stored in the storagethrough an input of the user. The command of the display apparatusis not limited thereto and may be various types of command.

The display apparatusindependently performs voice control inbut may operate along with the external server apparatus to perform the voice control.