Welcome to dbForumz.com!
FAQFAQ    SearchSearch      ProfileProfile    Private MessagesPrivate Messages   Log inLog in

Reports in OLTP system

 
   Database Forums (Home) -> Oracle RSS
Next:  index versus 'order by' question  
Author Message
Wiktor Moskwa

External


Since: Jul 28, 2007
Posts: 7



(Msg. 1) Posted: Sat Jul 28, 2007 4:27 pm
Post subject: Reports in OLTP system
Archived from groups: comp>databases>oracle>misc (more info?)

Hello,

Let's consider an accounting application for small businessess. It is
a web based application with about 2000 users issueing invoices and
making lots of other business transactions. In the same time they
want to view tax reports, generate tax declarations and see general
reports of how the business is doing.
Most users would like to see reports very frequently.

My question is what are best practices in this area. How to generate
reports for individual users based on source data that is changing
all the time.

My initial proposals are (database is Oracle 10R2):
1. Use triggers to maintain up-to-date reports
2. Use "fast refresh on commit" materialized view as reports
3. Manually generate a report in PL/SQL procedure when it is requested
by a user and is not up-to-date - otherwise present the one
calculated last time
4. Queue requests for reports and generate them in the background
(advenced queing would be used)

Triggers and materialized views will have impact on data modifications,
moreover I think triggers should not contain business logic and
materialized views are discouradged for OLTP systems.
In cases 3 and 4 a business transactions has to mark some reports
as old.

What can you suggest?
Thanks in advance!

--
Wiktor Moskwa

 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
DA Morgan

External


Since: May 16, 2005
Posts: 1301



(Msg. 2) Posted: Sat Jul 28, 2007 4:27 pm
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Wiktor Moskwa wrote:
> Hello,
>
> Let's consider an accounting application for small businessess. It is
> a web based application with about 2000 users issueing invoices and
> making lots of other business transactions. In the same time they
> want to view tax reports, generate tax declarations and see general
> reports of how the business is doing.
> Most users would like to see reports very frequently.
>
> My question is what are best practices in this area. How to generate
> reports for individual users based on source data that is changing
> all the time.
>
> My initial proposals are (database is Oracle 10R2):
> 1. Use triggers to maintain up-to-date reports
> 2. Use "fast refresh on commit" materialized view as reports
> 3. Manually generate a report in PL/SQL procedure when it is requested
> by a user and is not up-to-date - otherwise present the one
> calculated last time
> 4. Queue requests for reports and generate them in the background
> (advenced queing would be used)
>
> Triggers and materialized views will have impact on data modifications,
> moreover I think triggers should not contain business logic and
> materialized views are discouradged for OLTP systems.
> In cases 3 and 4 a business transactions has to mark some reports
> as old.
>
> What can you suggest?
> Thanks in advance!

I'd suggest taking a serious look at BAM.

http://www.oracle.com/technology/products/integration/bam/index.html

Why reinvent the wheel?
--
Daniel A. Morgan
University of Washington
damorgan.DeleteThis@x.washington.edu (replace x with u to respond)
Puget Sound Oracle Users Group
www.psoug.org

 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Wiktor Moskwa

External


Since: Jul 28, 2007
Posts: 7



(Msg. 3) Posted: Sat Jul 28, 2007 8:06 pm
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On 28.07.2007, DA Morgan wrote:
>
> I'd suggest taking a serious look at BAM.
>
> Why reinvent the wheel?

Well, I want to learn how such things are done.
Are materialized views or advanced queuing right tools
here or not.

I'd appreciate any advice.

--
Wiktor Moskwa
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
DA Morgan

External


Since: May 16, 2005
Posts: 1301



(Msg. 4) Posted: Sat Jul 28, 2007 9:59 pm
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Wiktor Moskwa wrote:
> On 28.07.2007, DA Morgan wrote:
>> I'd suggest taking a serious look at BAM.
>>
>> Why reinvent the wheel?
>
> Well, I want to learn how such things are done.
> Are materialized views or advanced queuing right tools
> here or not.
>
> I'd appreciate any advice.

To determine that would be a full scale research project. They might
be and then again they might not be. I know organizations doing it
with Data Guard replication to a logical standby so it very much
depends.

Give some metrics with respect to number of users, transaction size,
number of transactions, etc. and perhaps someone from a similarly
sized organization can tell you what worked or didn't work for them.
--
Daniel A. Morgan
University of Washington
damorgan.TakeThisOut@x.washington.edu (replace x with u to respond)
Puget Sound Oracle Users Group
www.psoug.org
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Wiktor Moskwa

External


Since: Jul 28, 2007
Posts: 7



(Msg. 5) Posted: Tue Jul 31, 2007 11:57 am
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On 29.07.2007, DA Morgan wrote:
> Wiktor Moskwa wrote:
>>
>> Well, I want to learn how such things are done.
>> Are materialized views or advanced queuing right tools
>> here or not.
>
> To determine that would be a full scale research project. They might
> be and then again they might not be. I know organizations doing it
> with Data Guard replication to a logical standby so it very much
> depends.

Thanks for a reply. Let me change my question a little.
I wonder what is a usual practice of serving aggregate data in
a standard database (not a warehouse, used to store current data).
By aggregate data I mean for example monthly sales summary.

The example environment is a web application with 2000 users
(each is a different <very> small business, not related to others,
with distinct subset of data). How to provide them with effective
up-to-date summaries? (triggers? MVs? queuing? calculation on
request and storing for later use?)

And a sub-question: with two tables INVOICE (parent) and
INVOICE_ITEM (child) - do you usually keep sum of values
from child rows in parent row or calculate in application
every time?

Thanks in advance.

--
Wiktor Moskwa
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
DA Morgan

External


Since: May 16, 2005
Posts: 1301



(Msg. 6) Posted: Tue Jul 31, 2007 3:52 pm
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Wiktor Moskwa wrote:
> On 29.07.2007, DA Morgan wrote:
>> Wiktor Moskwa wrote:
>>> Well, I want to learn how such things are done.
>>> Are materialized views or advanced queuing right tools
>>> here or not.
>> To determine that would be a full scale research project. They might
>> be and then again they might not be. I know organizations doing it
>> with Data Guard replication to a logical standby so it very much
>> depends.
>
> Thanks for a reply. Let me change my question a little.
> I wonder what is a usual practice of serving aggregate data in
> a standard database (not a warehouse, used to store current data).
> By aggregate data I mean for example monthly sales summary.
>
> The example environment is a web application with 2000 users
> (each is a different <very> small business, not related to others,
> with distinct subset of data). How to provide them with effective
> up-to-date summaries? (triggers? MVs? queuing? calculation on
> request and storing for later use?)
>
> And a sub-question: with two tables INVOICE (parent) and
> INVOICE_ITEM (child) - do you usually keep sum of values
> from child rows in parent row or calculate in application
> every time?
>
> Thanks in advance.

Depends on the SLA. What is your obligations with respect to security,
auditing, backup and recovery, etc.

My preference would be replication to a logical standby but finances
are always a factor.
--
Daniel A. Morgan
University of Washington
damorgan.DeleteThis@x.washington.edu (replace x with u to respond)
Puget Sound Oracle Users Group
www.psoug.org
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Wiktor Moskwa

External


Since: Jul 28, 2007
Posts: 7



(Msg. 7) Posted: Tue Aug 14, 2007 2:56 pm
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On 03.08.2007, joel garry wrote:
> On Jul 31, 8:09 am, Wiktor Moskwa wrote:
>> Thanks for a reply. Let me change my question a little.
>> I wonder what is a usual practice of serving aggregate data in
>> a standard database (not a warehouse, used to store current data).
>> By aggregate data I mean for example monthly sales summary.
>>
>> The example environment is a web application with 2000 users
>> (each is a different <very> small business, not related to others,
>> with distinct subset of data). How to provide them with effective
>> up-to-date summaries? (triggers? MVs? queuing? calculation on
>> request and storing for later use?)
>
> Triggers not so good. In the past, when hardware was much slower, I'd
> do the calculation on request with storing for later use. I don't
> think I'd do that nowadays, but with the faster hardware the wheel
> that's already installed is like instant virtual DW's. Partioning
> would perhaps be more appropriate, but then again, we're talking
> organizations that are too cheap to have a second machine for
> reporting or DW, much less pay big licensing bucks if they don't
> _have_ to.
>

Hi Joel,

Thanks for your advice. That's exactly how you guessed - second DB
or partitioning is not an option.
We don't have any performance problems at the moment but I'd like
to design new parts and redesign older parts of this app "in
a correct way". In the application that I've recently iherited
the biggest problems are:
1. A lot of business logic in triggers
2. Undocumented, unpackaged procedures, some of them called from
application, some used internally - big chaos; and most of them
had been written without thinking about concurrency...
3. Java object-relational mapping framework that "simplifies database
access and isolates you from the complexity of databases" which
practically means that you can't force it do anything useful.
Curse on it!

> Some people are fans of VPD, I'm still reserving judgement on that,
> given the limitations it adds. For the situations I see, at any rate.
>

I've never used it but for me the idea behind VPD fits quite
well into our environment - each user (a small business) "owns"
a disjoint part of a table. But it would be useful for security
reasons only I think and wouldn't solve our current problems.

>> And a sub-question: with two tables INVOICE (parent) and
>> INVOICE_ITEM (child) - do you usually keep sum of values
>> from child rows in parent row or calculate in application
>> every time?
>
> Calculate every time, except for a modification where they need a
> snapshot to be able to reprint what was printed previously (I added a
> table for that, since this is a packaged app that makes a big deal
> about the dynamism). There are a lot more than two tables involved.
>

Two tables were used for simplification. Of course an invoice has to
be recorded in few "files" (I'm not sure if it's a correct English
word for it). Is my guess correct that your application would invoke
a single stored procedure that implements whole transaction and puts
invoice's data into some tables and updates other tables?

That's at least how I'd do it - a facade of packaged procedures that do
all the business logic, queries executed by functions returning ref
cursor. But I wonder if it is a good idea to put virtually everything
into procedures so that application never issues anything else than
PL/SQL call. For example simple create/update/delete/browse involving
one table at the time would need procedures for all those actions for
every such table. What are "best practices" in this area?

Thanks for patience.

--
Wiktor Moskwa
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Gints Plivna

External


Since: Aug 15, 2007
Posts: 6



(Msg. 8) Posted: Wed Aug 15, 2007 12:26 am
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On 28 J l., 19:27, Wiktor Moskwa wrote:
> Hello,
>
> Let's consider an accounting application for small businessess. It is
> a web based application with about 2000 users issueing invoices and
> making lots of other business transactions. In the same time they
> want to view tax reports, generate tax declarations and see general
> reports of how the business is doing.
> Most users would like to see reports very frequently.
>
> My question is what are best practices in this area. How to generate
> reports for individual users based on source data that is changing
> all the time.
>
> My initial proposals are (database is Oracle 10R2):
> 1. Use triggers to maintain up-to-date reports
> 2. Use "fast refresh on commit" materialized view as reports
> 3. Manually generate a report in PL/SQL procedure when it is requested
> by a user and is not up-to-date - otherwise present the one
> calculated last time
> 4. Queue requests for reports and generate them in the background
> (advenced queing would be used)
>
> Triggers and materialized views will have impact on data modifications,
> moreover I think triggers should not contain business logic and
> materialized views are discouradged for OLTP systems.
> In cases 3 and 4 a business transactions has to mark some reports
> as old.
>
> What can you suggest?
> Thanks in advance!
>
> --
> Wiktor Moskwa

There are things you can do and things you cannot do in this world.
You cannot get the speed and power of Ferrari for the Opel price.
Users might want up to date reports but (depending on their complexity
of course) this might need additional boxes and/or other resources.
You haven't said BTW what are these reports? Are these simply some ~10
rows for the most recent transactions for a specific user that you can
got from 3 tables using indexes and nested loops or are these complex
aggregates needing for a full scan of ~10 tables?
If the latter and you haven't another box for reporting there are 3
possibilities:
1) calculate reports from business data and you can easily calculate
necessary time for it just running when there is no load on the system
2) calculate reports from some on commit refresahble Materialized
views and understand that this will add quite significant overhead on
your business transactions.
3) calculate reports from some periodically refreshable MVs and let
users understand that they actually a) cannot get most recent data
because of lack of horse powers b) most recent data on a busy system
many times are just unnecessary luxury - you start to genarate precise
report on 10.00 and end it say 10.02. But situation has been changed
and overall totals also have been changed. So you could easily get
along with data on 7.00 because the main goal usually for such reports
are some trends.

We had similar situation like you (need for complex searches) and we
simply understood that we cannot generate them from business data and
we cannot afford on commit MVs, so we got periodically refreshed MVs
and forced users to accept that. Simple - either pay for Ferrrari or
if you cannot, you can get Opel Wink
OK and my experience is described here - Effective search in a
normalized application at http://www.gplivna.eu/papers/mat_views_search.htm

Gints Plivna
http://www.gplivna.eu
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Wiktor Moskwa

External


Since: Jul 28, 2007
Posts: 7



(Msg. 9) Posted: Wed Aug 15, 2007 9:39 pm
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

On 15.08.2007, Gints Plivna wrote:
> [..]
> We had similar situation like you (need for complex searches) and we
> simply understood that we cannot generate them from business data and
> we cannot afford on commit MVs, so we got periodically refreshed MVs
> and forced users to accept that. Simple - either pay for Ferrrari or
> if you cannot, you can get Opel Wink
> OK and my experience is described here - Effective search in a
> normalized application at http://www.gplivna.eu/papers/mat_views_search.htm
>

Thanks for this case study, it was very informative.
I think I'll leave triggers as they are, at least for now - there
are no performance problems at the moment.

I liked your "Do you have data waste or data base?" artictle too -
I've seen few applications that accurately match this description.
Few weeks ago a friend of mine was repairing (or should I say
rebuilding) a database that had no single index - "that would be
a premature optimization" the other team said. On the other hand
most tables were partitioned by some irrevelant, surrogate id, into
over 200 partitions were most queries didn't have this id in WHERE
clause - "we've partitioned it because database contains a lot of data
and it will increase performance" they said. The db was PostgreSQL
and "a lot of data" was around 300 000 rows...

--
Wiktor Moskwa
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Gints Plivna

External


Since: Aug 15, 2007
Posts: 6



(Msg. 10) Posted: Thu Aug 16, 2007 5:06 am
Post subject: Re: Reports in OLTP system [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

> Few weeks ago a friend of mine was repairing (or should I say
> rebuilding) a database that had no single index -

Khekhe
Yeahh when I was in 2nd year in university we had to create a small DB
project and app on it. I've created home library in FoxPro2.6. My
teacher asked "Do you have any indexes?" I answered "No" and thought
"what a silly question - everything works for me!" Yeahh it had some
~100 records Wink
So the moral is - what is excusable for a dumb and inexperienced 2nd
year student isn't excusable for people building quite serious apps...

Gints Plivna
http://www.gplivna.eu
 >> Stay informed about: Reports in OLTP system 
Back to top
Login to vote
Display posts from previous:   
Related Topics:
Oracle Reports Win98 vs WinXP - We've just finished migrating from windows 98 to windows XP only to discover that there are printing issues with Oracle Reports 6i in Windows XP. We are using Dot matrix printers to print our invoices (we have tried several brands - Panasonic KX 1124i,....

DROP or MODIFY COLUMN on Oracle 8 system - Hi I need to change the datatype of a column from VARCHAR2 to FLOAT preserving all data, which actually represent numeric data. I have to do it on an oracle8 enterprise system. Afaik ALTER TABLE tablename DROP COLUMN column and / or ALTER TABLE..

How to read a file in the client system (Forms 6i) using9iAS - My requirement is i am having a file in the client machine. But the forms (Version 6i) is running on the 9iAS. Now i have to access the text file ie. i have to read that file and i have to upload that contents into a table. I am using the Text_IO package...

Hot opening for IT Manager(System Admin) in World Class Pr.. - Hi Allow me to take this opportunity to introduce to you our "TechUnified Consulting" has been adjudged as the "Best Emerging Company" for the year 2007.We are a premier HR consulting based in Bangalore, currently working with a larg...

Oracle view question - Hi, The source table has a column which is varchar2(100). If I create a view new_view on the column as select substr(column_name, 1, 20) new_col from table, then "desc new_view" command shows size of new_col as 60 i.e. 3 times the intended siz...
   Database Forums (Home) -> Oracle All times are: Pacific Time (US & Canada)
Page 1 of 1

 
You can post new topics in this forum
You can reply to topics in this forum
You can edit your posts in this forum
You can delete your posts in this forum
You can vote in polls in this forum



[ Contact us | Terms of Service/Privacy Policy ]