Web services (SOA) from a Web Application

R

richardsosborn

I'm working in a large enterprise architecture. We'll be accessing
web services
(probably via an ESB). I didn't see any posts on it. I'm looking
for integration
patterns to access these services from a web framework (Spring).
 
T

Tom Forsmo

I'm working in a large enterprise architecture. We'll be accessing
web services
(probably via an ESB). I didn't see any posts on it. I'm looking
for integration
patterns to access these services from a web framework (Spring).

I think you should perhaps get a book on Web Services to understand how
to do that, its not a just simple integration pattern. You have to
understand the ideas behind Web Services and the history of business
systems integration to understand how the idea of SOA can help you. SOA
and ESB are just marketing hype, what is really meant is that its
basically about how to integrate systems and what is really going to
help you solve you problem is understanding what I have mentioned above.

regards

tom
 
?

=?ISO-8859-1?Q?Arne_Vajh=F8j?=

I'm working in a large enterprise architecture. We'll be accessing
web services
(probably via an ESB). I didn't see any posts on it. I'm looking
for integration
patterns to access these services from a web framework (Spring).

I think you need 5 years of experience with Java, J2EE and
enterprise architecture.

Sorry for the hard words, but the wording in your posting
says that you do not have the experience necessary to design
what you are asking about.

Arne
 
B

BioInfoGuy

Arne said:
I think you need 5 years of experience with Java, J2EE and
enterprise architecture.

Sorry for the hard words, but the wording in your posting
says that you do not have the experience necessary to design
what you are asking about.

Arne

C'mon Arne,

Go easy on the guy. Many a junior programmer has made a great architect
and we ought not to presume otherwise merely by virtue of a good
question...

Jason
 
R

richardsosborn

BioInfoGuy said:
C'mon Arne,

Go easy on the guy. Many a junior programmer has made a great architect
and we ought not to presume otherwise merely by virtue of a good
question...

Jason

yea arne. because your perception was wrong. i have 11 years
experience.
i designed the frameworking publishing "washingonpost.com" as well
as frameworks for "novartis", "deutsche banc" and the "department of
homeland security".
i've worked in asychronous messaging (jms) but not sychronous (web
services).
can i get a little help?
 
D

Danno

yea arne. because your perception was wrong. i have 11 years
experience.
i designed the frameworking publishing "washingonpost.com" as well
as frameworks for "novartis", "deutsche banc" and the "department of
homeland security".
i've worked in asychronous messaging (jms) but not sychronous (web
services).
can i get a little help?


Hehe, you're clearly a n00b.

I can help a little with you are asking. I just want to point out the
difference between SOA and Web Services. It took me a while to figure
out what people are going nuts about. SOA is, get this.....a business
facade. I have been hearing business people and fellow software
engineers go overboard on the definition, but that's all it is, and it
has nothing to do with Web Services. It is just "re-engineering" your
code to be more business-centric. The funny thing is that good
enterprise developers do that any way, which I am sure you do.

Now the key to enterprise Web Services is to integrate a WebServices
layer without any intrusion into your enterprise system. The JavaEE 5
Stack is coming out with a simple annotation that you can place on your
stateless session beans. The Apache Tuscany Project does too. As per
Spring 2.0, it seems you have to integrate all the pieces
(http://static.springframework.org/spring/docs/2.0.x/reference/remoting.html).
I am sure that the Spring guys will come up with a great solution soon
for an all-in-one stack.
 
R

richardsosborn

Danno said:
Hehe, you're clearly a n00b.

I can help a little with you are asking. I just want to point out the
difference between SOA and Web Services. It took me a while to figure
out what people are going nuts about. SOA is, get this.....a business
facade. I have been hearing business people and fellow software
engineers go overboard on the definition, but that's all it is, and it
has nothing to do with Web Services. It is just "re-engineering" your
code to be more business-centric. The funny thing is that good
enterprise developers do that any way, which I am sure you do.

Now the key to enterprise Web Services is to integrate a WebServices
layer without any intrusion into your enterprise system. The JavaEE 5
Stack is coming out with a simple annotation that you can place on your
stateless session beans. The Apache Tuscany Project does too. As per
Spring 2.0, it seems you have to integrate all the pieces
(http://static.springframework.org/spring/docs/2.0.x/reference/remoting.html).
I am sure that the Spring guys will come up with a great solution soon
for an all-in-one stack.


this is what i was wondering. i've done EAI. i understand SOA fine.
i just don't know how to connect a web application to this. most
enterprise
code i've done was written around an existing enterprise application in
the
first place. and most of that was in JMS.

i see spring supports "services". that sounds like a good integration
point,
since it will probably be transactional. just wondering what to do if
they chose
struts, or something else. my thought right now is to use the service
support in
spring. if they go struts, create a session facade and expose a SSB
(EJB) as a web service.
 
T

Tom Forsmo

this is what i was wondering. i've done EAI. i understand SOA fine.
i just don't know how to connect a web application to this. most
enterprise
code i've done was written around an existing enterprise application in
the
first place. and most of that was in JMS.

i see spring supports "services". that sounds like a good integration
point,
since it will probably be transactional. just wondering what to do if
they chose
struts, or something else. my thought right now is to use the service
support in
spring. if they go struts, create a session facade and expose a SSB
(EJB) as a web service.

Could you be a bit more clear on what you are asking about and also what
the architecture of the existing system is. That would make it a lot
easier to help you.

for example, you mention jms, is that the interface of the existing
systems? you also mention spring "services" are you talking about spring
support for web services/soap, jms or web app. And kind of client are
you looking to connect, a web app?

regards

tom
 
R

richardsosborn

Tom said:
Could you be a bit more clear on what you are asking about and also what
the architecture of the existing system is. That would make it a lot
easier to help you.

for example, you mention jms, is that the interface of the existing
systems? you also mention spring "services" are you talking about spring
support for web services/soap, jms or web app. And kind of client are
you looking to connect, a web app?

regards

tom

right now, we're in design talks. the design calls for web
applications to connect to an enterprise service bus. this could
deliver both web service and jms message calls.
with the web framework being the client, it shouldn't really matter.
i'm looking for the nuts and bolts of how these will talk to each
other. i'm assuming you'd have to call from the business or service
layer of your web framework, do a lookup, start the transaction, and
call the service on the ESB.

spring seems to have it's own framework for accessing services. from
what i read
it is transactional. i would simply do a lookup on the ESB, start
spring's transaction, then
pass my web service query. j2ee also allows exposing an ejb as a web
service. i'm not sure how easily this could be rolled to work in the
SOA paradigm. i could also easily just write a stateless session bean
(ejb) and have it access the ESB. it could start off the transaction,
do the lookup on the ESB and pass the query. using spring seems the
most simple.

my last resort is to mimic a "data access object" pattern to create a
"service object
access" pattern. like dao, i would have high level objects for the
general services.
they would do the lookup against the ESB, began a JTA transaction. i
would then have finer grained classes represent the queries which will
be passed to the jms queues or web services.

i wasn't sure if anyone had done any of this before and had best
practices to share for all of us. i'm sure we'll be going through
iterations of trial and testing. i'll gladly share the results.
(except to arne).
 
M

Manish Pandit

Hi!

SOA brings a new paradigm of business awareness. Along with looking at
the technology, make sure you consider the cross-functional business
processes. Under the hood, SOA is essentially
identify->assemble->deploy cycle, where in you identify common
cross-functional tasks, model them as services, assemble multiple
services (tasks) to achieve a coarse grained functionality (process),
and deploy it. The core focus should be granularity and reusability -
the code can be EJBs, Spring Components or a mainframe sitting
somewhere. This brings in the whole Enterprise Integration scenario,
where in you might end up integrating some legacy systems via messaging
or something similar. Normally, businesses do not prefer "rewrite the
whole thing" idea, so there is always going to be some wrapping.
Speaking of technology, start with very common utilities that can be
reused across applications (like audit/logging/service invocation...)
so that you will have a framework to begin with. Along with identifying
services, focus on message specs (input/output XML schemas), security
etc. Implementing SOA is a pretty big effort, given the
cross-functional and multi-tiered nature of business applications.

Best of luck and do keep us posted!

-cheers,
Manish
 
R

richardsosborn

Manish said:
Hi!

SOA brings a new paradigm of business awareness. Along with looking at
the technology, make sure you consider the cross-functional business
processes. Under the hood, SOA is essentially
identify->assemble->deploy cycle, where in you identify common
cross-functional tasks, model them as services, assemble multiple
services (tasks) to achieve a coarse grained functionality (process),
and deploy it. The core focus should be granularity and reusability -
the code can be EJBs, Spring Components or a mainframe sitting
somewhere. This brings in the whole Enterprise Integration scenario,
where in you might end up integrating some legacy systems via messaging
or something similar. Normally, businesses do not prefer "rewrite the
whole thing" idea, so there is always going to be some wrapping.
Speaking of technology, start with very common utilities that can be
reused across applications (like audit/logging/service invocation...)
so that you will have a framework to begin with. Along with identifying
services, focus on message specs (input/output XML schemas), security
etc. Implementing SOA is a pretty big effort, given the
cross-functional and multi-tiered nature of business applications.

Best of luck and do keep us posted!

-cheers,
Manish


our business processes and business design are all laid out. in this
project, next phase is software design and implementation. assuming we
can integrate web applications and SOA services fine, re-use of part of
all of any particular service should be no problem. that motivation is
actually driving some of the software design. how do we design to
easily re-leverage part or all of an existing service? how do we
design to allow this to happen quickly and simply? it looks like,
apart from vendor's docs and best practices, we're on our own.
 
T

Tom Forsmo

>
right now, we're in design talks. the design calls for web
applications to connect to an enterprise service bus. this could
deliver both web service and jms message calls.
with the web framework being the client, it shouldn't really matter.
i'm looking for the nuts and bolts of how these will talk to each
other. i'm assuming you'd have to call from the business or service
layer of your web framework, do a lookup, start the transaction, and
call the service on the ESB.

spring seems to have it's own framework for accessing services. from
what i read
it is transactional. i would simply do a lookup on the ESB, start
spring's transaction, then
pass my web service query. j2ee also allows exposing an ejb as a web
service. i'm not sure how easily this could be rolled to work in the
SOA paradigm. i could also easily just write a stateless session bean
(ejb) and have it access the ESB. it could start off the transaction,
do the lookup on the ESB and pass the query. using spring seems the
most simple.

my last resort is to mimic a "data access object" pattern to create a
"service object
access" pattern. like dao, i would have high level objects for the
general services.
they would do the lookup against the ESB, began a JTA transaction. i
would then have finer grained classes represent the queries which will
be passed to the jms queues or web services.

i wasn't sure if anyone had done any of this before and had best
practices to share for all of us. i'm sure we'll be going through
iterations of trial and testing. i'll gladly share the results.
(except to arne).

I think I better understand what you are looking for now.

To respond to your request, if I understand you correctly. I think the
way to do it is to use an integration tier at the client side. The
integration tier would be responsible for presenting an abstract
interface of the operations the ESB provides which the web app requires.
A tier below the integration tier would be the actual communication
mechanism used, e.g. jms, esb, web services etch.

This would give you a three tiered client (webapp/integration/comm) that
would allow you to create the desired technology mix in the client. E.g.
all tiers could use spring or the webapp could use struts and the other
two tiers could use spring and so on. Features such as transactions is
handled at the communication tier by way of signal from the integration
tier, i.e. implicit in method or by an argument.
Depending on how the communication is designed in the system, you could
possibly use an enterprise message format which travels from the client
integration tier to the service integration tier instead of an explicit
service interface.

This solution, though, is architecture independent, the reason for this
is that I am not a great believer in SOA and ESB. I would love, though,
to hear what you choose to do and how you solve the issues, as there are
not many actually doing SOA in real life.

(Sorry, the following explanation is quite long, but it is not easy to
condense)

In any case, as terms, SOA and ESB, are just "syntactic sugar"/business
terms for the age-old business component model (corba, dcom etc) of the
nineties. Its using the well know concept of service brokering to
present an idea of component/service/utility computing. I don't think
the component model works at a business level, mainly because business
processes are specialised processes for a special case in a business
that management creates to give their business an "competitive edge".

What I mean by this is that any business process/step is usually so
specialised, that it in most cases it can not be made into a general
components (or as termed in SOA: service/utility etc). The principle of
business components is easy to accept, but the reality is not. Any
business process steps requires specialised input/processing/output that
can not easily be generalised and reused in a service chain of other
processes or contexts without adaptation, this making it less generalised.

I will explain in detail how I see it, if I have missed something, I
trust you will share your experience.

For example, lets say you have a telco which has a business process
called "New ADSL customer", which consists of the following simplified
steps

1) gather customer data
2) check existence of customer
2b) create new customer
3) check credit rating
4) check availability
5) order service personel/date
6) set subscription and details
7) activate subscription in exchange

The general description of the steps in this process seems fair enough
and could possibly make you think it can be turned into components. But
the details of each step, what they require as arguments and what they
produce as output are different for each type of "service" used. What 1)
receives is dependent on the product being bought. 2) might need
different arguments depending on the product or system it need to talk
with, which 1 might not have gathered. what information 2b) requires to
register a new customer depends on the product and 2). There might also
be steps in 2b that have to be performed in the middle of 1 to perform
correctly which are only required by 1 in certain products. 3),4) and 5)
could possibly be components, but it depends. 6) depends on 1 and the
product type. Finally 7) depends on the specific information about the
product type, the type of exchange and so on. So there are a lot of
dependencies on the product which would suggest that you actually need
specialised steps for each product type, which it how its done normally.

Regarding an ESB I don't agree on the need. Any communication can be
performed by exposing any enterprise systems through the systems SOAP
interface. A few operations, in a few systems, will probably be reused
by many systems, but mostly there will be an 1-on-1 integration between
a client and a service. By using SOAP, the communication layer is
independent. And by designing the interfaces as an organic interface
(i.e. an xml document where arguments and operations are specified so
that it can be changed without breaking backwards compatibility or
future expansions), one can change implementations/technologies in the
client or service without the system breaking.

But as I said, I am quite curious as to how you see it an how you choose
to solve it. So please let me know.

tom
 
R

richardsosborn

What I mean by this is that any business process/step is usually so
specialised, that it in most cases it can not be made into a general
components (or as termed in SOA: service/utility etc). The principle of
business components is easy to accept, but the reality is not. Any
business process steps requires specialised input/processing/output that
can not easily be generalised and reused in a service chain of other
processes or contexts without adaptation, this making it less generalised.

I will explain in detail how I see it, if I have missed something, I
trust you will share your experience.

For example, lets say you have a telco which has a business process
called "New ADSL customer", which consists of the following simplified
steps

1) gather customer data
2) check existence of customer
2b) create new customer
3) check credit rating
4) check availability
5) order service personel/date
6) set subscription and details
7) activate subscription in exchange

The general description of the steps in this process seems fair enough
and could possibly make you think it can be turned into components. But
the details of each step, what they require as arguments and what they
produce as output are different for each type of "service" used. What 1)
receives is dependent on the product being bought. 2) might need
different arguments depending on the product or system it need to talk
with, which 1 might not have gathered. what information 2b) requires to
register a new customer depends on the product and 2). There might also
be steps in 2b that have to be performed in the middle of 1 to perform
correctly which are only required by 1 in certain products. 3),4) and 5)
could possibly be components, but it depends. 6) depends on 1 and the
product type. Finally 7) depends on the specific information about the
product type, the type of exchange and so on. So there are a lot of
dependencies on the product which would suggest that you actually need
specialised steps for each product type, which it how its done normally.

Regarding an ESB I don't agree on the need. Any communication can be
performed by exposing any enterprise systems through the systems SOAP
interface. A few operations, in a few systems, will probably be reused
by many systems, but mostly there will be an 1-on-1 integration between
a client and a service. By using SOAP, the communication layer is
independent. And by designing the interfaces as an organic interface
(i.e. an xml document where arguments and operations are specified so
that it can be changed without breaking backwards compatibility or
future expansions), one can change implementations/technologies in the
client or service without the system breaking.

But as I said, I am quite curious as to how you see it an how you choose
to solve it. So please let me know.

tom

thanks for all of the input. like i've said, i'm very familiar with
EAI design and
concepts and usage in SOA. i come from a 90's background in it.
this sounds very similar to the design i'm preparing for presentation
prior to
moving into development.

i could agree ESB might be a little overkill, but from the large
integrations that could
take place enterprise-wide, i see some of their motivation.

we will be choosing and cross-sharing the services "ala carte" once we
have some in
place. we have most business operations abstracted to this level (IE
"enter content",
"evaluate content") and some are mapped by the role of the person using
them.
the thought is hopefully the web application client will "partake" of
some portions
of one service and some of another one. this is the motivation behind
SOA. we'll see if it works in practice. i'll keep the thread updated.

thanks again for the input.
 
T

Tom Forsmo

we will be choosing and cross-sharing the services "ala carte" once we
have some in
place. we have most business operations abstracted to this level (IE
"enter content",
"evaluate content")

This is the part I am curious about how you see it be done.
So I would like to explore it further and I hope you can share your
thoughts on how you see this so I can understand better.

- Are "enter content", "evaluate content" and other similar operations
the defined business operations made available by the ESB?
- Is this the level you see operations being made available in the SOA
architecture you are making? (in contrast to the more usual operations
"add customer", "delete customer" etc)
and some are mapped by the role of the person using
them.

Are you saying here some of the abstracted operation you have designed
are not machine operations but human operations?
the thought is hopefully the web application client will "partake" of
some portions
of one service and some of another one.

Do I understand you correctly here if I say that to create an externally
available service (i.e. one used by clients of the SOA, say a web page),
the web app would use all of these business operations in t mix to
create the final webapp? For example made up by using BPEL, to design
the webapp?
i'll keep the thread updated.

please do, I am very interested in how this turns out.

tom
 
R

richardsosborn

Tom said:
This is the part I am curious about how you see it be done.
So I would like to explore it further and I hope you can share your
thoughts on how you see this so I can understand better.

- Are "enter content", "evaluate content" and other similar operations
the defined business operations made available by the ESB?
- Is this the level you see operations being made available in the SOA
architecture you are making? (in contrast to the more usual operations
"add customer", "delete customer" etc)


Are you saying here some of the abstracted operation you have designed
are not machine operations but human operations?


Do I understand you correctly here if I say that to create an externally
available service (i.e. one used by clients of the SOA, say a web page),
the web app would use all of these business operations in t mix to
create the final webapp? For example made up by using BPEL, to design
the webapp?


please do, I am very interested in how this turns out.

tom

you're exactly right. some of the operations we've set up are human
operations, i guess in a sense. they're not mapped to classes or
layers but business operations performed internally. the system
architect has abstracted the available services into chunks which
ideally will be re-usable, based on future needs. my concern is making
this as easy to do as possible. let me give an example.

he's designed "evaluate content", "enter content" etc into services.
if my coding layer to them is just as flexible, it should be possible
to re-leverage services and portions of services over and over. this
should be relatively easy. that's my concern.

i was hoping more people had done this, given the timestamp of some of
the earlier SOA messages. but we'll see how it goes. should be in
software design next week.
 
T

Tom Forsmo

you're exactly right. some of the operations we've set up are human
operations, i guess in a sense. they're not mapped to classes or
layers but business operations performed internally. the system
architect has abstracted the available services into chunks which
ideally will be re-usable, based on future needs. my concern is making
this as easy to do as possible. let me give an example.

he's designed "evaluate content", "enter content" etc into services.
if my coding layer to them is just as flexible, it should be possible
to re-leverage services and portions of services over and over. this
should be relatively easy. that's my concern.

If that is the level of detail the design of the services are aimed at,
I have one serious concern. It will require an excessive amount of
network capacity, probably to the level that operations in full
production will be so slow that no one is going to use it in the end.
This is of course, if I the understand the design correctly.

The problem I see is that all these services are made available on the
ESB. The number of services has to be quite large due to the small-size
design of each individual service. When designing an end service, with
f.ex BPEL, you are chaining lots of smaller services into a larger
end-service, which is going to have to communicate through the ESB. In a
more traditional system, most of that communication would happen within
a single server, i.e. in memory. Which is hundred, if not thousand,
times faster. With ESB, you have network latency and network speed which
affects the overall performance. So, add to this a full scale production
load. with smaller and larger operations running. There will be so many
messages going around on the ESB that the ESB will be overloaded. An
end-service might end up taking 10, 20 or 30 minutes to finish.

I worked in a an enterprise project some years ago where they did
something similar, they went for a object oriented design on how to
communicate with the DB. A single abstract "get customer dial plans"
operation took 20 minutes to complete, and that was with only a few
clients. So they changed the design, to have the DB return all data in
one request. Then it only took 10-20 seconds.

tom
 
R

richardsosborn

If that is the level of detail the design of the services are aimed at,
I have one serious concern. It will require an excessive amount of
network capacity, probably to the level that operations in full
production will be so slow that no one is going to use it in the end.
This is of course, if I the understand the design correctly.

The problem I see is that all these services are made available on the
ESB. The number of services has to be quite large due to the small-size
design of each individual service. When designing an end service, with
f.ex BPEL, you are chaining lots of smaller services into a larger
end-service, which is going to have to communicate through the ESB. In a
more traditional system, most of that communication would happen within
a single server, i.e. in memory. Which is hundred, if not thousand,
times faster. With ESB, you have network latency and network speed which
affects the overall performance. So, add to this a full scale production
load. with smaller and larger operations running. There will be so many
messages going around on the ESB that the ESB will be overloaded. An
end-service might end up taking 10, 20 or 30 minutes to finish.

I worked in a an enterprise project some years ago where they did
something similar, they went for a object oriented design on how to
communicate with the DB. A single abstract "get customer dial plans"
operation took 20 minutes to complete, and that was with only a few
clients. So they changed the design, to have the DB return all data in
one request. Then it only took 10-20 seconds.

tom

i think that drove the design of the "value object" design pattern. ;)
i'm getting pretty low level, but another concern is transformation.
it eats CPU power grotesquely. converting all of the objects or
information
from one format to another is the most costly expense on a CPU.

for anyone interested, i found an ebook that directly relates to
integrating
within SOA. it's one of ibm's "redbooks" and it's called "patterns:
service-oriented
architecture and web services". it divides leveraged patterns into
groups depending
on the service it will expose.
 
R

richardsosborn

i think that drove the design of the "value object" design pattern. ;)
i'm getting pretty low level, but another concern is transformation.
it eats CPU power grotesquely. converting all of the objects or
information
from one format to another is the most costly expense on a CPU.

for anyone interested, i found an ebook that directly relates to
integrating
within SOA. it's one of ibm's "redbooks" and it's called "patterns:
service-oriented
architecture and web services". it divides leveraged patterns into
groups depending
on the service it will expose.


i'll expand more on this since a few seem interested in it, and there
wasn't much discussion
previously. martin fowler's "patterns of enterprise architecture"
talks about a number of helps integrating enterprise services. one is
a "service layer", which i'm modeling up.

he speaks of using stateless session beans to interface web services or
whatever other remote service you might use. despite the serious
downplay of ejb lately, there are good uses for it. they are rare but
enterprise integration (which i believe they were designed for) is one
of them. it's nice to leverage the transaction support, exposure as
web services or a jms component. with ejb 3.0, the code is easier than
ever to write.

fowler's "data transfer object" and "domain model" are also good pieces
to use in your enterprise integration effort.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,769
Messages
2,569,579
Members
45,053
Latest member
BrodieSola

Latest Threads

Top